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EXECUTIVE  SUMMARY 


Issue  and  Overview 

This  report  describes  the  current  state  of  instructional  supervision  in  Special  Operations  Forces  (SOF) 
initial  acquisition  training  (1AT)  language  schools  and  compares  the  instructional  supervisory  behaviors 
and  practices  used  in  these  schools  to  those  identified  in  the  theoretical,  empirical,  and  best  practices 
literature  on  this  topic,  as  well  as  to  those  used  in  language  schools  external  to  the  SOF  1AT  community. 
Using  the  information  presented,  SOF  1AT  leadership,  Command  Language  Program  Managers 
(CLPMs),  and  instructional  supervisors  can  maintain  and  further  develop  effective  instructional 
supervisory  behaviors  and  practices  in  SOF  1AT  language  schools. 

Method 

A  thorough  review  of  the  theoretical,  empirical,  and  best  practices  literature  in  the  fields  of  education, 
psychology,  second  language  acquisition,  and  training  was  conducted  to  identify  effective  instructional 
supervisory  behaviors  and  skills.  Following  the  literature  review,  both  SOF  1AT  current-state  and 
external  benchmarking  studies  were  conducted  to  identify  instructional  supervisory  behaviors  and 
practices  currently  used  in  SOF  1AT  language  schools  and  in  external  language  schools,  respectively. 
The  information  collected  from  the  literature  review  and  both  the  SOF  1AT  current-state  and  external 
benchmarking  studies  were  used  to  identify  gaps  between  the  current  and  desired  states  of  instructional 
supervision  in  SOF  1AT  language  schools  and  provide  SOF  1AT  leadership,  CLPMs,  and  instructional 
supervisors  with  recommendations  for  closing  these  gaps  (Figure  1,  p.  2). 

Figure  1.  Gaps  between  Current  State  of  and  Best  Practices  for  Instructional  Supervision 
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Findings 

Based  on  the  literature  review,  effective  instructional  supervisors: 

•  Perform  a  variety  of  administrative  or  general  supervisory  behaviors  outside  of  the  classroom. 
These  include  staffing,  scheduling  and  planning,  budgeting,  reporting,  and  applying 
organizational  policy. 

•  Perform  a  variety  of  classroom  or  clinical  supervisory  behaviors  inside  the  classroom,  such  as 
conducting  classroom  observations  of  instructors’  performance  and  providing  performance 
evaluations  and  feedback  to  instructors  after  observations  are  conducted.  Effective  instructional 
supervisors  also  hold  instructors  accountable  for  failing  to  meet  performance  standards. 

•  Engage  in  professional  development  activities,  provide  opportunities  for  instructors’  professional 
development,  coach  and  mentor  instructors,  and  openly  and  honestly  communicate  with 
instructors.  Both  group  process  and  effective  communication  skills  enable  instructional 
supervisors  to  engage  more  effectively  in  these  behaviors. 

•  Hold  Master’s  or  Doctoral  degrees  in  second  language  acquisition  or  instruction,  education,  or 
other  relevant  fields  and  have  relevant  prior  experience  in  administration,  instruction,  and  in 
military  environments  or  organizations. 

The  findings  from  the  SOF  1AT  current-state  study  were  organized  into  five  main  categories:  (1)  language 
learners  and  language  learning  environment,  (2)  position  of  instructional  supervisor,  (3)  relationships 
between  instructional  supervisors  and  instructors,  (4)  strategies  and  methods  for  instructional  supervision, 
and  (5)  classroom  observations  and  assessments.  Some  of  the  main  findings  included: 

Language  Learners  and  Language  Learning  Environment 

•  In  general,  language  learners  in  SOF  1AT  language  schools  were  similar  to  language  learners  in 
external  schools  in  terms  of  their  native  language  and  previous  language  experience.  SOF  1AT 
language  learners  differed  from  external  language  learners  in  terms  of  their  gender  and  purpose 
for  language  learning. 

•  Although  the  languages  offered  in  external  language  schools  were  similar  to  languages  currently 
taught  within  the  SOF  1AT  community,  course  durations  were  shorter  and  class  meetings  were 
less  frequent  than  in  SOF  1AT  language  schools.  In  external  language  schools,  there  was 
substantial  variability  in  terms  of  how  often  students’  proficiency  was  assessed. 

Position  of  Instructional  Supervisor 

•  In  both  SOF  1AT  and  external  language  schools,  instructional  supervisors  have  a  diverse  set  of 
responsibilities.  Although  differences  were  observed  between  SOF  1AT  language  schools  and 
between  these  language  schools  and  external  language  schools,  there  were  several  commonalities, 
such  as  monitoring  instructor  performance,  providing  constructive  feedback  to  instructors,  and 
providing  resources  to  instructors. 
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•  Common  instructional  supervisory  qualifications  included  cultural  expertise,  English  proficiency, 
instructional  experience,  and  administrative  experience. 

Relationships  between  Instructional  Supervisors  and  Instructors 

•  The  number  of  instructional  supervisors  in  SOF  1AT  language  schools  ranged  from  1-5,  with  as  a 
few  as  five  instructors  to  as  many  as  35  instructors  per  supervisor.  The  range  of  responses  from 
external  language  schools  was  too  large  to  be  comparable  to  SOF  1AT  language  schools. 

•  Participants  from  both  SOF  1AT  and  external  language  schools  reported  similar  challenges  in 
working  with  instructors.  These  included  instructors’  unwillingness  to  learn  or  cooperate,  lack  of 
cultural  understanding,  and  inexperience  in  education  and  teaching. 

Strategies  and  Methods  for  Instructional  Supervision 

•  Effective  instructional  supervisors  in  both  the  SOF  1AT  and  external  language  schools  engaged  in 
professional  development  activities  and  provided  opportunities  for  instructors’  professional 
development. 

•  Supervisors  in  both  SOF  1AT  and  external  language  schools  encouraged  their  instructors  to  use 
particular  models  of  instruction  in  the  classroom.  The  most  commonly  mentioned  models  were 
communicative,  task-based,  and  learner-centered. 

•  A  difference  was  observed  between  SOF  1AT  language  schools  and  external  language  schools  in 
terms  of  the  role  supervisors  play  in  instructor  motivation.  Only  two  external  language  program 
administrators  indicated  instructional  supervisors  in  their  schools  play  a  role  in  instructor 
motivation. 

Classroom  Observations  and  Assessments 

•  Instructional  supervisors  in  both  SOF  1AT  and  external  language  schools  engage  in  a  number  of 
practices  related  to  classroom  observations  and  assessments.  These  practices  differ  somewhat 
between  SOF  1AT  language  schools  and  between  these  schools  and  external  language  schools. 

Recommendations 

The  following  recommendations  are  provided  to  SOF  LAT  leadership  and  CLPMs  to  advance  the 

theoretical,  empirical,  and  best  practices  literature  on  instructional  supervision  in  adult,  military-related 

instructional  contexts,  such  as  SOF  1AT  language  schools. 

•  Task  and  knowledge,  skills,  and  abilities  (KSA)  analyses  should  be  conducted  in  each  of  the  SOF 
1AT  language  schools  to  better  understand  the  tasks  instructional  supervisors  perform  in  these 
language  schools  and  to  empirically  identify  the  KSAs  required  for  them  to  be  effective  in  their 
performance  of  these  tasks.  By  conducting  these  task  and  KSA  analyses,  SOF  1AT  leadership 
and  CLPMs  can  better  ensure  supervisor  selection,  training,  and  development  interventions  are 
aligned  with  the  task  and  KSA  requirements  of  instructional  supervisory  positions  in  SOF  1AT 
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language  schools  and  that  instructional  supervisors  have  the  resources  they  need  to  perform  their 
jobs  effectively  (cf.  Surface,  2012). 

•  Additional  research  should  be  conducted  to  better  understand  how  effective  instructional 
supervision  in  SOF  1AT  language  schools  is  related  to  instructor  effectiveness  and  student 
learning  outcomes,  such  as  language  proficiency.  This  research  would  extend  our  understanding 
of  how  supervision  in  adult,  military-related  instructional  contexts  is  related  to  instructor 
effectiveness  and  student  learning  outcomes.  To  facilitate  this  research,  additional  data  on 
instructional  supervision  in  SOF  1AT  language  schools  should  be  collected. 

Based  on  the  findings  of  the  SOF  1AT  current-state  study  and  external  benchmarking  studies,  the 
following  recommendations  are  provided  to  SOF  1AT  leadership  and  CLPMs  to  maintain  and  further 
develop  effective  instructional  supervisory  behaviors  and  practices  in  SOF  1AT  language  schools: 

•  Flire  or  promote  candidates  with  the  qualifications  necessary  to  be  effective  instructional 
supervisors.  This  includes  selecting  instructional  supervisors  based  on  education,  instructional 
experience,  administrative  experience,  and  English  proficiency. 

•  Create  an  on-the-job  training  program  for  instructional  supervisors.  Among  other  topics,  this 
training  program  should  be  focused  on  effective  interpersonal  communication  skills,  such  as 
active  listening,  administrative  skills,  and  the  SOF  1AT  instructional  context. 

•  Provide  necessary  resources  to  instructional  supervisors  to  remove  barriers  to  supervisory 
effectiveness.  Examples  include  providing  support  staff  to  reduce  the  ratio  of  instructors  to 
supervisors  in  some  SOF  1AT  language  schools  and  providing  resources  to  support  supervisors’ 
and  instructors’  professional  development. 

Based  on  the  findings  of  the  SOF  1AT  current-state  and  external  benchmarking  studies,  the  following 
recommendations  are  provided  to  current  instructional  supervisors  to  maintain  and  further  develop  their 
effectiveness: 

•  Gain  instructor  buy-in  in  observation,  evaluation,  and  feedback  processes  to  maximize  the 
effectiveness  of  these  processes.  Instructor  buy-in  can  be  gained  by  meeting  with  instructors 
prior  to  conducting  observations,  conducting  more  frequent  observations,  and  providing  feedback 
from  multiple  sources. 

•  Supervisors  should  leverage  existing  resources,  such  as  Instructor  Feedback  Reports  (IFRs)  and 
the  Instructor  Developmental  Feedback  Guide,  to  provide  instructors  with  multi-source  feedback 
and  recommendations  for  improvement.  An  1FR  training  module  should  be  developed  to  help 
supervisors  understand  IFRs  and  how  they  can  be  used. 

•  Make  an  effort  to  understand  instructors’  languages  and  cultures  when  they  are  different  than 
one’s  own.  Learn  greetings  and  commonly  used  phrases  in  these  languages  to  build  rapport  with 
instructors. 

•  Communicate  openly  and  honestly  with  instructors  to  assure  instructors  their  input  is  valued. 


9/28/2012 


©  SWA  Consulting  Inc.,  2012 
Technical  Report  [2012010632] 


Page  5 


SOFLO  Support  Project 


Best  Practices  for  Supervision  of  Instructors 


Caveats 

In  interpreting  the  findings  from  the  theoretical,  empirical,  and  best  practices  literature  review  on  this 
topic,  as  well  as  the  findings  from  both  the  SOF  IAT  current-state  and  external  benchmarking  studies,  the 
reader  is  encouraged  to  consider  a  few  important  caveats  regarding  these  findings: 

•  Much  of  the  literature  on  this  topic  is  theoretical  in  nature  and  focuses  on  supervision  in  K-12 
instructional  contexts.  As  such,  the  reader  is  encouraged  to  consider  how  the  instructional 
supervisory  behaviors,  skills,  and  qualifications  reported  in  the  literature  might  differ  in  adult, 
military-related  instructional  contexts,  such  as  SOF  IAT  language  schools. 

•  Current  SOF  IAT  language  program  vendors  were  included  as  participants  in  the  SOF  IAT 
current-state  study.  If  language  program  vendors  change  in  any  or  all  of  the  SOF  IAT  language 
schools,  then  the  instructional  supervisory  behaviors  and  practices  reported  in  Section  IV  (pp.  23- 
43)  may  be  subject  to  change. 

•  Although  all  SOF  IAT  language  program  administrators  and  current  language  program  vendors 
were  invited  to  participate  in  the  current-state  study,  all  of  them  were  not  able  to  participate. 
Likewise,  only  eight  of  the  35  external  language  schools  contacted  participated  in  the  external 
benchmarking  study.  Thus,  the  findings  presented  in  this  report  may  be  subject  to  change  if 
additional  interviews  are  conducted. 

This  project  was  conducted  by  SWA  Consulting  Inc.  under  a  subcontract  with  CACI-WGI,  Inc. 
(Subcontract#  B1 1-1 14482;  Prime#  H92222-10-D-001 7/0007;  Sub-CLIN  0003 AB).  For  questions  or 
more  information  about  the  SOFLO  and  this  project,  please  contact  Mr.  Jack  Donnelly 
(john. donnelly@s oc om. mil).  For  specific  questions  related  to  data  collection  or  this  report,  please  contact 
Dr.  Eric  A.  Surface  (esurface@swa-consulting.com)  or  Dr.  Reanna  Poncheri  Harman  ( rpharman@s  wa- 
consulting.com). 
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SECTION  I:  REPORT  PURPOSE  AND  OVERVIEW 
Establish  Best  Practices  for  Supervision  of  Instructors  Report  Purpose 

Many  elements  of  Special  Operations  Forces  (SOF)  foreign  language  training  may  be  leveraged  to 
improve  training  outcomes.  One  element  is  the  effective  supervision  of  SOF  initial  acquisition  training 
(1AT)  foreign  language  instructors.  Supervisors  of  foreign  language  instructors  in  the  SOF  1AT 
community  have  a  responsibility  to  support  effective  instruction,  which  in  turn  supports  improved  student 
learning  and  training  outcomes.  In  the  case  of  SOF  1AT,  these  efforts  will  support  the  development  of 
language-capable  SOF  personnel.  This  report  describes  the  current  state  of  instructional  supervision  in 
SOF  1AT  language  schools  and  compares  the  supervisory  behaviors  and  practices  used  in  these  schools  to 
effective  instructional  supervisory  behaviors  and  practices  identified  in  the  theoretical,  empirical,  and  best 
practices  literature  on  this  topic,  as  well  as  to  those  used  in  language  schools  external  to  the  SOF  1AT 
community.  Using  the  information  presented,  SOF  1AT  leadership.  Command  Language  Program 
Managers  (CLPMs),  and  instructional  supervisors  can  maintain  and  further  develop  effective  instructional 
supervisory  behaviors  and  practices  in  SOF  IAT  language  schools. 

Establish  Best  Practices  for  Supervision  of  Instructors  Report  Overview 

This  report  contains  the  following  sections  that  describe  effective  instructional  supervisory  behaviors  and 
practices  used  in  SOF  IAT  language  schools  and  compare  these  behaviors  and  practices  to  those 
described  in  the  theoretical,  empirical,  and  best  practices  literature  on  this  topic.  In  addition,  instructional 
supervisory  behaviors  and  practices  used  in  language  schools  external  to  the  SOF  IAT  community  are 
also  described. 

•  Section  11  (pp.  9-10)  describes  why  effective  instructional  supervision  is  important  to  developing 
more  language-capable  SOF  personnel. 

•  Section  111  (p.  11-22)  defines  supervision  and  summarizes  the  theoretical,  empirical,  and  best 
practices  literature  on  effective  instructional  supervision  to  identify  effective  instructional 
supervisory  behaviors,  skills,  and  qualifications. 

•  Section  IV  (p.  23-43)  describes  the  current  state  of  instructional  supervision  in  SOF  IAT  language 
schools.  This  section  also  describes  the  results  of  an  external  benchmarking  study  conducted 
with  language  schools  external  to  the  SOF  IAT  community. 

•  Section  V  (p.  44-50)  provides  an  evaluation  and  synthesis  of  both  the  SOF  IAT  current-stale  and 
external  benchmarking  studies  to  identify  gaps  between  the  current  and  desired  states  for 
effective  instructional  supervision  in  SOF  IAT  language  schools.  This  section  includes 
recommendations  for  SOF  IAT  leadership,  CLPMs,  and  current  instructional  supervisors  in  the 
SOF  IAT  community. 
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SECTION  II:  IMPORTANCE  OF  EFFECTIVE  INSTRUCTIONAL  SUPERVISION 

Effective  instructional  supervision  is  necessary  to  develop  language-capable  SOF  personnel.  For  SOF 
operators  to  develop  their  language  proficiency  during  SOF  1AT,  operators  must  be  trained  by  effective 
foreign  language  instructors  who  are  developed  and  supported  by  effective  instructional  supervisors.  This 
section  emphasizes  the  importance  of  effective  instructional  supervision  to  developing  language-capable 
SOF  personnel  in  SOF  1AT  language  schools. 

In  the  SOF  community,  operators  must  develop  and  maintain  the  language  capability  needed  to  meet  their 
language-related  mission  requirements  and  to  achieve  mission  success  (United  States  Special  Operations 
Command  [USSOCOM]  M350-8).  Despite  the  importance  of  developing  language-capable  SOF 
personnel,  barriers  to  language  acquisition  and  maintenance  do  exist  for  SOF  operators.  For  example, 
SOF  operators  experience  other  training  requirements  that  compete  with  language  training  ( Barriers  to 
Language  Acquisition  and  Maintenance,  Technical  Report  #2010011024).  Thus,  because  SOF  1AT 
training  time  is  limited,  it  is  critically  important  that  all  elements  of  SOF  1AT  be  leveraged  to  mitigate 
barriers  to  SOF  operators’  language  acquisition  and  maintenance.  One  such  element  is  the  effective 
supervision  of  SOF  1AT  foreign  language  instructors. 

While  much  less  is  known  about  how  effective  instructional  supervision  is  related  to  both  instructor 
effectiveness  and  student  learning  outcomes  in  the  SOF  community,  preliminary  evidence  suggests 
instructional  supervisors  in  SOF  1AT  language  schools  do,  indeed,  impact  instructor  effectiveness.  Using 
training  effectiveness  data  collected  from  1AT  students  and  instructors  at  the  U.S.  Army  John  F.  Kennedy 
Special  Warfare  Center  and  School  (SWCS)  between  2010-2012,  data-analytic  results  revealed  the 
following: 

•  Instructors  who  reported  ever  receiving  an  Instructor  Feedback  Report  (1FR) 1,2  received  higher 
Semester  2  instructor  evaluation  ratings  from  their  students  on  Engage  (?(23 1.78)  =  4. 16,  p  < 
.000),  Manage  (# 222.20)  =  5.75,  p  <  .000),  Respond  (#234.35)  =  4.70,  p  <  .000),  and  Adapt 
(1(231.37)  =  4.46,  p  <  .000),  the  four  areas  of  instructor  effectiveness  assessed  in  the  IFRs.  With 
the  exception  of  Adapt,  these  findings  were  similar  for  Semester  3  instructor  evaluation  ratings. 

•  Instructors  who  reported  receiving  their  Semester  1  IFRs  received  higher  Semester  2  instructor 
evaluation  ratings  from  their  students  on  Engage  (#579.69)  =  1 .99,  p  <  .05),  Manage  (#559.71)  = 
2.68, p  <  .01),  Respond  (#598.82)  =  2.10 ,p<  .05),  and  Adapt  (#579.21)  =  2.24, p  <  .05). 

Thus,  given  that  instructional  supervisors  play  a  role  in  distributing  IFRs  to  the  instructors  they  supervise 
and,  more  generally,  in  providing  performance  feedback  to  instructors,  this  preliminary  evidence  suggests 
instructor  effectiveness  is  impacted  by  instructional  supervisory  effectiveness. 

Despite  the  limited  empirical  evidence  linking  instructional  supervisory  effectiveness  to  student  learning 
outcomes,  empirical  findings  regarding  the  relationship  between  instructor  effectiveness  and  student 

1  IFRs  are  formative  training  evaluation  reports  provided  by  SWA  Consulting  Inc.  to  language  program 
administrators,  language  contract  vendors,  and  instructors  in  the  SOF  language  training  community.  These  reports 
identify  instructors’  strengths  and  areas  for  development  during  training  and  provide  information  for  action  at  all 
levels,  including  a  system  for  continuous  instructor  self-guided  development. 

2  This  survey  item  asked  instructors  if  they  had  ever  received  an  IFR  at  some  point  during  their  tenure  at 
USAJFKSWCS. 
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performance  can  be  leveraged  to  shed  additional  light  on  how  instructional  supervisory  effectiveness  may 
impact  student  learning  outcomes.  For  example,  Muijs  and  Reynolds  (2003)  found  that  effective 
instruction  explained  up  to  24%  of  the  variance  in  student  performance  in  academic  environments. 
Likewise,  in  the  SOF  community,  nearly  50%  of  SOF  operators’  performance  on  the  Defense  Language 
Proficiency  Test  (DLPT)  was  explained  by  instructor  effectiveness  (SWA  Consulting  Inc.,  2009, 
Technical  Report  #2009010602).  Taken  together,  these  findings  underscore  the  importance  of  effective 
foreign  language  instruction  to  developing  language-capable  SOF  personnel.  By  extension,  instructional 
supervisors  play  a  key  role  in  developing  language-capable  SOF  personnel  because  they  develop  effective 
foreign  language  instructors  in  SOF  1AT  language  schools. 

In  Section  111  (pp.  11-22),  the  theoretical,  empirical,  and  best  practices  literature  on  instructional 
supervision  is  summarized  to  identify  effective  instructional  supervisory  behaviors,  skills,  and 
qualifications.  These  behaviors,  skills,  and  qualifications  were  then  used  to  generate  interview  questions 
to  conduct  a  current-state  study  of  instructional  supervision  in  SOF  1AT  language  schools,  as  well  as  an 
external  benchmarking  study  conducted  with  language  schools  external  to  the  SOF  1AT  community 
(Section  IV,  pp.  23-43). 
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SECTION  III:  BEST  PRACTICES  FOR  EFFECTIVE  INSTRUCTIONAL  SUPERVISION 


Purpose 

A  thorough  review  of  the  theoretical,  empirical,  and  best  practices  literature  in  the  fields  of  education, 
psychology,  second  language  acquisition,  and  training  was  conducted.  The  purpose  of  this  review  was  to 
identify  effective  instructional  supervisory  behaviors,  skills,  and  qualifications.  Although  most  of  the 
literature  on  this  topic  is  theoretical  in  nature,  emphasis  was  placed  on  studies  that  examined  the  effects  of 
instructional  supervision  on  key  outcomes,  such  as  instructor  effectiveness.  Additionally,  a  review  of 
SWA  Consulting  Inc.’s  (SWA)  previous  research  and  evaluation  efforts  for  the  Special  Operations  Forces 
Language  Office  (SOFLO)  was  also  conducted.  These  sources  were  then  summarized,  and  this 
information  was  used  to  generate  a  list  of  effective  instructional  supervisory  behaviors,  skills,  and 
qualifications.  From  this  list,  four  main  themes  were  identified:  (1)  administrative  or  general  supervision, 
(2)  classroom  or  clinical  supervision,  (3)  other  effective  instructional  supervisory  behaviors  and  skills, 
and  (4)  instructional  supervisory  qualifications. 

Research  Questions 


•  What  effective  instructional  supervisory  behaviors  are  performed  by  instructional  supervisors 
outside  of  the  classroom? 

•  What  effective  instructional  supervisory  behaviors  are  performed  by  instructional  supervisors 
inside  the  classroom? 

•  What  additional  behaviors  and  skills  enable  instructional  supervisors  to  be  effective  in  their  roles 
both  outside  of  and  inside  the  classroom? 

•  What  qualifications  (e.g.,  education,  experience)  enable  instructional  supervisors  to  be  effective 
in  their  roles  both  outside  of  and  inside  the  classroom? 


Main  Findings 

Effective  instructional  supervisors: 

•  Perform  a  variety  of  administrative  or  general  supervisory  behaviors  outside  of  the  classroom. 
These  behaviors  include  staffing,  scheduling  and  planning,  budgeting,  reporting,  and  applying 
organizational  policy. 

•  Perform  a  variety  of  classroom  or  clinical  supervisory  behaviors  inside  the  classroom,  such  as 
conducting  classroom  observations  of  instructors’  performance  and  providing  performance 
evaluations  and  feedback  to  instructors  after  observations  are  conducted.  Effective  instructional 
supervisors  also  hold  instructors  accountable  for  failing  to  meet  performance  standards. 

•  Engage  in  professional  development  activities,  provide  opportunities  for  instructors’  professional 
development,  coach  and  mentor  instructors,  and  openly  and  honestly  communicate  with 
instructors.  Both  group  process  and  effective  communication  skills  enable  instructional 
supervisors  to  engage  more  effectively  in  these  behaviors. 
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•  Hold  Master’s  or  Doctoral  degrees  in  second  language  acquisition  or  instruction,  education,  or 
relevant  fields  and  have  relevant  prior  experience  in  administration,  instruction,  and  in  military 
environments  or  organizations. 

Recommendations 

•  Task  and  knowledge,  skills,  and  abilities  (KSA)  analyses  should  be  conducted  in  each  of  the  SOF 
1AT  language  schools  to  better  understand  the  tasks  instructional  supervisors  perform  in  these 
language  schools  and  to  empirically  identify  the  KSAs  required  for  them  to  be  effective  in  their 
performance  of  these  tasks.  KSAs  may  be  defined  as  (Gatewood,  Field,  &  Barrick,  2008): 

o  Knowledge  -  information  applied  directly  to  perform  an  essential  job  function. 

o  Skill  -  observable  competence  to  perform  a  job  task. 

o  Ability  -  competence  to  perform  an  observable  behavior  or  a  behavior  that  will  result  in 
an  observable  product  (e.g.,  written  report). 

By  conducting  these  task  and  KSA  analyses,  SOF  1AT  leadership  and  CLPMs  can  better  ensure 
supervisor  selection,  training,  and  development  interventions  are  aligned  with  the  task  and  KSA 
requirements  of  instructional  supervisory  positions  in  SOF  1AT  language  schools  and  that 
instructional  supervisors  have  the  resources  they  need  to  perform  their  jobs  effectively  (cf. 
Surface,  2012). 

•  Additional  research  should  be  conducted  to  better  understand  how  effective  instructional 
supervision  in  SOF  1AT  language  schools  is  related  to  instructor  effectiveness  and  student 
learning  outcomes,  such  as  language  proficiency.  Given  that  much  of  the  literature  on  this  topic 
is  theoretical  in  nature  and  focuses  on  supervision  in  K-12  instructional  contexts,  this  research 
would  extend  our  understanding  of  supervision  in  adult,  military-related  instructional  contexts, 
such  as  SOF  1AT  language  schools. 

Literature  Review 

The  review  of  the  theoretical,  empirical,  and  best  practices  literature  in  the  fields  of  education, 
psychology,  second  language  acquisition,  and  training  resulted  in  the  identification  of  four  main  themes 
regarding  effective  instructional  supervisory  behaviors,  skills,  and  qualifications:  (1)  administrative  or 
general  supervision,  (2)  classroom  or  clinical  supervision,  (3)  other  effective  instructional  supervisory 
behaviors  and  skills,  and  (4)  instructional  supervisory  qualifications.  The  main  findings  for  each  of  these 
themes  are  described  in  detail  below.  In  addition,  a  definition  of  supervision  is  provided  to  aid  the 
reader’s  understanding  of  what  supervision  is  and  how  it  is  commonly  defined  in  instructional  contexts. 

What  is  supervision? 

Management,  or  supervision,  can  be  defined  as  “the  attainment  of  organizational  goals  in  an  effective  and 
efficient  manner  through  planning,  organizing,  staffing,  directing,  and  controlling  organizational 
resources”  (Daft,  2008,  p.  14).  More  specifically,  instructional  supervision  is  focused  on  the 
improvement  of  instruction  and  learning  outcomes  (Gebhard,  1990;  Glanz,  Schulman,  &  Sullivan,  2007; 
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Lunenberg,  1998;  Neagley  &  Evans,  1980;  Sergiovanni  &  Starratt,  1998)  and  can  be  further  divided  into 
two  categories:  (1)  administrative  (“general”)  supervision,  and  (2)  classroom  (“clinical”)  supervision, 
with  the  former  concerned  with  supervisory  behavior  occurring  outside  of  the  classroom,  and  the  latter 
focusing  on  supervisory  behavior  inside  the  classroom  (Neagley  &  Evans,  1980;  Wallace,  1991).  The 
discussion  that  follows  focuses  first  on  administrative  supervisory  behaviors. 

Administrative  or  General  Supervision 

Administrative  or  general  supervision  involves  supervisory  behavior  that  generally  occurs  outside  of  the 
classroom  (Neagley  &  Evans,  1980;  Wallace,  1991).  Although  these  administrative  behaviors  do  not 
constitute  the  full  set  of  behaviors  in  which  instructional  supervisors  must  engage  to  be  effective,  these 
behaviors  do  enable  supervisors  to  form  a  strong  foundation  for  effective  instructional  supervision.  In 
other  words,  effective  administrative  supervision  is  necessary,  but  not  sufficient,  for  effective 
instructional  supervision.  Administrative  behaviors  may  include  staffing,  scheduling  and  planning, 
budgeting,  reporting  and  applying  organizational  policy. 

Staffing 

Effective  instructional  supervisors  are  involved  in  the  process  of  staffing  foreign  language  instructors,  to 
the  extent  possible. 

In  the  SOF  IAT  context,  instructional  supervisors  may  play  a  role  in  recruiting  and  selecting  foreign 
language  instructors  of  sufficient  quantity  and  quality  to  support  the  United  States  Special  Operations 
Command  (USSOCOM)  in  its  goal  to  develop  language-capable  SOF  personnel  (USSOCOM  M350-8). 
Staffing  is  defined  as  “the  process  of  acquiring,  deploying,  and  retaining  a  workforce  of  sufficient 
quantity  and  quality  to  create  positive  impacts  on  the  organization's  effectiveness"  (Fleneman,  Judge,  & 
Kammeyer-Mueller,  2012,  p.  8).  Depending  on  the  structure  of  the  organization,  instructional  supervisors 
in  SOF  IAT  language  schools  may  have  varying  amounts  of  involvement  in  the  staffing  process  for 
foreign  language  instructors.  To  the  extent  possible,  instructional  supervisors  should  be  actively 
involved,  as  this  may  have  a  positive  impact  on  the  performance  of  their  workgroup.  In  one  study, 
supervisors’  involvement  in  effective  staffing  processes  accounted  for  14%  of  the  variance  in  group 
performance  (Bare,  1978).  A  number  of  factors  could  explain  this  finding,  including  the  hiring  of 
qualified  and  experienced  personnel. 

Scheduling  and  Planning 

Effective  instructional  supervisors  schedule  and  plan  important  meetings  and  events. 

For  example,  instructional  supervisors  may  be  tasked  with  planning  and  scheduling  time  for  instructor 
training  and  professional  development  activities.  Likewise,  instructional  supervisors  may  deploy 
instructors  with  the  requisite  target  language  proficiency  and  skill  sets  to  teach  appropriate  courses. 

These  supervisors  may  also  schedule  time  to  meet  one-on-one  or  in  small  groups  with  foreign  language 
instructors  on  a  regular  basis  to  communicate  important  information  and  support  instructors’  professional 
development  efforts  by  providing  instructors  with  resources  and  performance  feedback. 
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Budgeting  and  Managing  Resources 

Effective  instructional  supervisors  manage  and  administer  budgets  for  some  or  all  aspects  of  their 
language  programs. 

These  budgets  may  cover  expenses  such  as  the  costs  of  professional  development  opportunities  for 
instructors,  as  well  as  supplemental  course  materials.  In  managing  and  administering  budgets,  effective 
instructional  supervisors  work  to  distribute  resources  equally  among  instructors  and  other  key  personnel 
(National  Association  of  Independent  Schools  [NAIS],  1990).  In  addition,  effective  instructional 
supervisors  know  where  to  locate  and  how  to  obtain  important  resources  (Humphrey  &  Stokes,  2000). 
These  resources  might  include  classroom  materials,  such  as  pens  or  whiteboard  markers,  or  course 
materials  that  are  used  to  supplement  the  curriculum  for  a  particular  language. 

Reporting 

Effective  instructional  supervisors  report  important  information  to  language  program  leadership  and  key 
stakeholders,  using  multiple  communication  media. 

These  reports  may  include  important  information  about  language  program  activities  (e.g.,  classroom 
observations  of  instructors),  outcomes  (e.g.,  students’  proficiency  scores),  and  trends  to  key  stakeholders, 
such  as  language  program  leadership,  language  program  administrative  staff,  and  contract  vendor 
leadership.  Although  these  reports  may  be  provided  through  multiple  communication  media,  written 
reports  may  be  the  medium  most  commonly  used  by  instructional  supervisors  for  reporting. 

Consistently  Applying  Policies 

Effective  instructional  supervisors  are  knowledgeable  about  and  consistently  apply  both  language 
program  and  organizational  policies  across  situations  and  instructors. 

Policy-related  decisions  may  include,  but  are  not  limited  to,  determining  leave  time  for  instructors,  such 
as  for  religious  holidays,  administering  performance-related  awards  to  instructors,  and  determining  how 
professional  development  opportunities  should  be  provided  to  instructors.  In  all  of  these  situations, 
effective  instructional  supervisors  communicate  and  explain  policies  to  instructors  at  the  beginning  of 
their  tenure  and  attempt  to  apply  policies  fairly  and  consistently  across  situations  and  the  instructors  they 
supervise  (NAIS,  1990).  In  doing  so,  instructional  supervisors  can  avoid  the  appearance  of  favoritism. 

Summary 

Collectively,  administrative  supervision  represents  one  category  of  behaviors  that  must  be  performed  by 
instructional  supervisors  to  be  effective.  These  behaviors  include  staffing,  scheduling  and  planning, 
budgeting  and  managing  resources,  reporting,  and  consistently  applying  both  program  and  organizational 
policies  across  situations  and  instructors.  These  behaviors  provide  an  important  foundation  for  the  second 
category  of  behaviors  in  which  instructional  supervisors  must  engage:  classroom  or  clinical  supervision. 
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Classroom  or  Clinical  Supervision 

Classroom  or  clinical  supervision  involves  supervisory  behaviors  that  generally  occur  inside  the 
classroom.  This  behavioral  category  is  typically  focused  on  “improving  instruction  and  learning  through 
the  direct  observation  of  teachers  and  pupils  in  action  in  the  learning  environment”  (Neagley  &  Evans, 
1980,  p.  20;  Wallace,  1991).  The  primary  instructional  supervisory  behaviors  that  constitute  classroom 
supervision  are  conducting  classroom  observations  of  instructors’  performance  and  providing 
performance  evaluations  and  feedback  to  instructors  after  observations  are  conducted. 

Observation,  Evaluation  and  Feedback:  A  Visual  Depiction 

Observation,  performance  evaluation,  and  feedback  are  effective  instructional  supervisory  practices  that 
are  well-documented  in  the  theoretical  literature  on  this  topic.  Lee  (2010)  integrated  these  concepts  into  a 
theoretical  model  known  as  the  Consultative  Supervision  and  Evaluation  (CS&E)  model.  According  to 
Lee,  if  the  steps  in  this  model  are  implemented  in  an  instructional  environment,  they  reinforce  effective 
instructional  supervisory  behavior  and  facilitate  instructor  engagement  in  and  awareness  of  their  progress 
towards  important  performance  goals. 

This  model  (Figure  2,  p.  16)  demonstrates  that  frequent  monitoring  of  instructors’  performance  is 
essential  for  effective  instructional  supervision.  This  monitoring  is  broken  into  two  phases:  (1) 
supervision  and  (2)  evaluation.  The  supervision  phase  is  focused  on  improving  classroom  instruction, 
while  the  evaluation  phase  is  more  focused  on  rating  instructors  for  quality  and  promotion  purposes  (Lee, 
2010).  In  both  cases,  the  model  assumes  supervision  and  evaluation  will  be  frequent  enough  to  provide 
instructors  with  specific,  actionable  suggestions  that  will  improve  the  quality  of  their  instruction.  As  is 
reflected  in  the  center  of  the  model,  the  relationships  between  instructional  supervisors  and  their 
instructors  form  the  core  of  this  process. 
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Figure  2.  Consultative  Supervision  and  Evaluation  (CS&E)  Model,  adapted  from  Lee  (2010) 
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Observation 

Effective  instructional  supervisors  conduct  classroom  observations  of  instructors  ’performance. 

Before  effective  instructional  supervisors  evaluate  or  give  feedback  to  instructors,  they  conduct  classroom 
observations  (Lee,  2010).  To  begin  the  process,  pre-observation  conferences  are  held  between 
instructional  supervisors  and  each  instructor  they  supervise.  The  purpose  of  these  meetings  is  to  build 
rapport  between  instructional  supervisors  and  instructors,  as  well  as  to  discuss  specific  areas  of 
performance  to  be  observed  and  evaluated.  Instructional  supervisors  and  instructors  should  agree  on  the 
criteria  against  which  instructors  will  be  evaluated,  and  additionally,  instructors  should  be  made  aware 
that  these  criteria  could  be  used  to  evaluate  their  performance  in  the  classroom  on  any  number  of  future 
announced  and  unannounced  classroom  observations  (Marshall,  2005). 

Upon  completion  of  the  pre-observation  conferences,  instructional  supervisors  observe  instructors  in  the 
classroom.  During  these  observations,  supervisors  collect  as  much  useful  information  about  the 
instructors’  performance  as  possible  to  generate  the  most  fair  and  accurate  evaluations.  In  addition, 
supervisors  take  detailed  notes  regarding  what  the  instructors  and  students  said  and  did  in  the  classroom 
(e.g.,  amount  of  target  language  speaking;  Sidhu  &  Fook,  2010).  After  these  observations,  supervisors 
promptly  schedule  post-observation  conferences  with  each  instructor  to  discuss  his  or  her  strengths  and 
areas  for  development  discovered  during  the  observations  (Gebhard,  1990;  Sidhu  &  Fook,  2010).  Once 
complete,  follow-up  conferences  are  scheduled  between  supervisors  and  instructors  to  discuss  next  steps 
for  performance  improvement. 

Evaluation 

Effective  instructional  supervisors  provide  performance  evaluations  to  instructors  after  classroom 
observations  are  conducted. 

While  similar  to  observation,  evaluation,  the  second  key  component  of  the  CS&E  model,  is  the  process  of 
forming  judgments  based  on  direct  observations  of  instructor  performance  (Lee,  2010).  To  mitigate  the 
potential  for  negative  instructor  attitudes  towards  the  performance  evaluation  process,  effective 
instructional  supervisors  provide  frequent  evaluations  of  instructors  and  if  their  time  is  limited,  they  enlist 
the  assistance  of  other  administrative  staff  members  to  provide  these  evaluations. 

Feedback 

Effective  instructional  supervisors  provide  both  positive  and  constructive  feedback  to  instructors,  focus 
on  behavior,  rather  than  personal  characteristics  or  traits,  when  providing  feedback,  involve  instructors 
in  the  feedback  process,  and  deliver  feedback  to  instructors  in  a  timely  manner. 

The  third  key  component  of  the  CS&E  model  is  feedback  (Lee,  2010).  According  to  Cleveland,  Lim,  and 
Murphy  (2007),  “The  only  task  more  difficult  than  receiving  performance  feedback  is  giving  performance 
feedback”  (p.  170).  While  the  feedback  process  may  seem  complex  and  anxiety-provoking,  effective 
instructional  supervisors  engage  in  a  number  of  practices  to  simplify  and  enhance  the  feedback  process. 
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Types  of  Feedback  Provided 

The  objective  of  the  feedback  provided  by  instructional  supervisors  to  instructors  should  be  to  improve 
instructor  performance.  Accordingly,  effective  instructional  supervisors  provide  both  positive  and 
constructive  feedback  to  instructors,  with  constructive  feedback  delivered  as  an  opportunity  for  learning 
(NA1S,  1990;  Ninomiya,  1988).  Both  during  the  feedback  process  and  throughout  instructors’  tenure, 
effective  instructional  supervisors  provide  positive  feedback  when  high  instructor  performance  is 
observed.  Approximately  80%  of  this  feedback  should  be  positive,  and  instructor  accomplishments 
should  be  discussed  upfront  in  feedback  delivery  meetings  (Berry,  Cadwell,  &  Fehrmann,  1996). 

In  many  instances,  instructional  supervisors  will  also  have  constructive  feedback  to  provide  to  instructors. 
These  areas  for  performance  improvement  should  be  discussed  after  positive  feedback  and 
accomplishments  are  discussed.  Effective  instructional  supervisors  provide  constructive  feedback  in  a 
manner  that  respects  instructors’  abilities  (NA1S,  1990).  When  delivered  effectively,  this  feedback 
develops  trust  and  rapport  between  supervisors  and  instructors  (Klein  &  Posey,  1986). 

When  providing  constructive  feedback  to  instructors,  effective  instructional  supervisors  focus  on 
instructor  behavior,  as  opposed  to  personality  characteristics  or  traits;  it  may  be  easier  for  feedback 
recipients  to  focus  on  what  they  did  or  did  not  do,  rather  than  who  they  are  or  are  not,  when  identifying 
areas  for  performance  improvement  (Harms  &  Roebuck,  20 1 0). 

Instructor  Involvement  in  Feedback  Process 

Feedback  has  traditionally  been  considered  a  unidirectional  process,  through  which  supervisors  provide 
suggestions  for  improvement  to  their  employees;  however,  modem  feedback  systems  involve  employees 
as  active  participants  in  the  feedback  process,  allowing  them  to  voice  their  opinions  and  collaboratively 
set  goals  (Norris,  1991). 

In  the  foreign  language  instructional  context,  instructors  may  desire  to  participate  in  the  feedback  process 
(Norris,  1991).  Accordingly,  effective  instructional  supervisors  involve  instructors  in  this  process.  This 
practice  is  supported  by  Ellermeyer  (1992),  who  suggests  instructors  should  be  involved  in  all  aspects  of 
the  feedback  process,  including  the  setting  of  metrics  and  goals  against  which  instructors  will  be 
evaluated.  This  consultative  process,  known  as  participatory  supervision  or  value-expressive  feedback, 
can:  (1)  enable  instructors  to  feel  more  comfortable  about  sharing  their  opinions  regarding  their 
performance,  (2)  build  instructor  ownership  in  the  feedback  process,  (3)  allow  for  greater  acceptance  of 
feedback  by  instructors,  and  (4)  enhance  perceptions  of  fairness  regarding  the  feedback  provided  by 
supervisors  (Cawley,  Keeping,  &  Levy,  1998;  Lizzio,  Wilson,  &  MacKay,  2008;  Schonberger,  2001). 

One  way  effective  instructional  supervisors  encourage  instructor  participation  in  the  feedback  process  is 
by  giving  instructors  an  opportunity  to  reflect  on  their  performance  prior  to  formal  feedback  meetings. 
These  self-reflections  also  allow  instructors  to  gather  their  thoughts  ahead  of  time,  rather  than  thinking 
about  them  during  feedback  meetings,  which  can  hinder  their  ability  to  listen  to  supervisor  feedback 
(Harms  &  Roebuck,  20 1 0). 
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Feedback  Delivery 

Effective  instructional  supervisors  keep  the  amount  of  time  between  classroom  observations  and  formal 
feedback  meetings  as  minimal  as  possible,  with  an  ideal  turnaround  time  of  24  hours.  Effective 
supervisors  also  attempt  to  frequently  deliver  less  formal  feedback  to  instructors. 

Summary 

Classroom  or  clinical  supervision  represents  another  set  of  behaviors  that  must  be  performed  by 
instructional  supervisors  to  be  effective.  These  behaviors  include  conducting  classroom  observations  of 
instructors’  performance  and  providing  performance  evaluations  and  feedback  to  instructors  after 
observations  are  conducted.  When  performed  regularly  and  effectively,  these  behaviors  provide  ongoing 
opportunities  for  supervisors  to  support  effective  instruction  in  the  classroom. 

Other  Effective  Instructional  Supervision  Behaviors  and  Skills 

Along  with  administrative  and  classroom  supervisory  behaviors,  effective  instructional  supervisors  also 
engage  in  several  other  behaviors,  such  as:  (1)  engaging  in  professional  development  activities,  (2) 
providing  opportunities  for  instructors’  professional  development,  (3)  coaching  and  mentoring  instructors, 
(4)  and  openly  and  honestly  communicating  with  instructors.  Both  group  process  and  effective 
communication  skills  enable  instructional  supervisors  to  engage  more  effectively  in  these  behaviors. 

Professional  Development 

Effective  instructional  supervisors  engage  in  professional  development  activities  and  provide 
opportunities  for  instructors’  professional  development. 

Supervisors’  professional  development  activities  may  involve  reading  articles,  books,  or  websites  to  stay 
abreast  of  recent  advances  in  their  fields.  They  may  also  attend  conferences,  short  courses,  or  workshops 
(Waters,  Marzano,  &  McNulty,  2003).  Supervisors  also  develop  themselves  professionally  by 
networking  with  other  professionals  or  by  joining  professional  organizations,  such  as  the  American 
Council  on  the  Teaching  of  Foreign  Languages  (ACTFL).  Collectively,  these  activities  help  supervisors 
remain  knowledgeable  about  and  up-to-date  on  recent  developments  in  their  fields. 

Much  like  supervisors,  instructors  can  benefit  by  attending  professional  events  and  reading  relevant 
research  and  best  practices  literature.  To  the  extent  possible,  effective  instructional  supervisors  provide 
instructors  with  opportunities  for  professional  development.  This  may  include  funding  professional 
development  workshops  or  training  for  instructors  or  disseminating  information  to  instructors  about 
recent  research  findings  or  best  practices. 

Coaching  and  Mentoring 

Effective  instructional  supervisors  coach  and  mentor  instructors. 

According  to  Gratton  (2008),  “One  of  the  most  crucial  organizational  levers  in  the  creation  of  cooperative 
working  environments  and  collaborative  teams  is  managers  who  coach  and  mentor  others  by  providing 
constant  feedback  on  employee  performance  (p.  9).”  In  the  foreign  language  instructional  context, 
effective  instructional  supervisors  maintain  an  open-door  policy  and  are  available  when  instructors  need 
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to  discuss  challenges  they  are  facing  or  concerns  they  have.  Further,  they  share  their  expertise  and 
resources  with  instructors  to  improve  instructors’  effectiveness  in  the  classroom.  For  example,  if  an 
instructor  is  experiencing  classroom  management  issues,  his  or  her  supervisor  can  provide  suggestions 
and  discuss  potential  solutions  for  remedying  these  issues.  In  the  SOF  1AT  context,  in  particular, 
instructional  supervisors  can  also  help  instructors  understand  the  nuances  of  instructing  in  a  military 
context,  a  context  with  which  many  instructors  may  be  inexperienced  and  unfamiliar. 

Open  and  Honest  Communication 

Effective  instructional  supervisors  openly  and  honestly  communicate  with  instructors. 

Instructors  may  be  more  likely  to  accept  and  internalize  feedback  in  environments  where  open  and  honest 
discussion  is  encouraged.  To  that  end,  effective  instructional  supervisors  foster  this  type  of  environment 
by  openly  and  regularly  communicating  important  information  to  the  instructors  they  supervise. 

Group  Process  Skills 

Effective  instructional  supervisors  possess  group  process  skills. 

Group  factors,  such  as  size  and  span  of  supervisor  control,  can  significantly  impact  supervisor-instructor 
relationships.  One  psychological  concept  known  as  span  of  attention  states  that  the  number  of  things  a 
person  can  typically  think  about  at  one  time  is  about  seven  (Entwisle  &  Walton,  1961).  This  concept 
suggests  that  for  a  supervisor  to  effectively  manage  a  group  of  instructors,  the  number  of  people  he  or  she 
is  responsible  for  should  be  restricted  to  that  approximate  number.  This  finding  is  supported  by  Gittell 
(2001),  who  suggests  that  smaller  group  sizes  are  most  effective  because  more  time  is  available  to  be 
spent  with  each  instructor.  Although  a  supervisor’s  span  of  control  may  be  out  of  his  or  her  control, 
effective  supervisors  are  aware  of  how  span  of  control  can  affect  their  relationships  with  instructors. 

Additionally,  effective  supervisors  facilitate  cooperative  and  non-conffontational  relationships  among 
their  supervisees  (Ninomoya,  1988;  Waters  et  al.,  2003).  Effective  supervisors  do  this  through  situational 
awareness  and  communication.  Situational  awareness  involves  being  sensitive  to  the  moods  and  attitudes 
of  instructors,  aware  of  informal  groups  and  relationships  among  instructors,  and  aware  of  any  potential 
issues  that  could  create  discord,  such  as  cultural  differences  (Ninomoya,  1988;  Waters  et  al.,  2003). 

Effective  Communication  Skills 

Effective  instructional  supervisors  possess  effective  communication  skills. 

Effective  supervisors  are  effective  communicators.  This  means  they  are  able  to  send  and  receive 
information  in  a  way  that  it  is  understood  by  all  parties  involved  (Hughes,  Ginnett,  &  Curphy,  2012; 
Pankake,  Stewart,  &  Winn,  1990.  To  do  this,  effective  supervisors  facilitate  two-way  communication 
(Pankake  et  al.,  1990).  In  addition,  they  are  aware  of  factors  such  as  varying  experiences  and  values  that 
could  affect  instructors’  understanding  and  tailor  their  communication  with  instructors  accordingly 
(Hughes  et  al.,  2012).  This  may  be  especially  challenging  and  important  in  SOF  IAT  environments, 
where  instructional  supervisors  and  instructors  may  not  share  the  same  primary  language  or  culture. 

Effective  supervisors  are  also  active  listeners  (Daft,  2008;  Hughes  et  al.,  2012;  Thomas,  1974).  Active 
listening  involves  focusing  on  what  is  being  said  and  verifying  the  message  was  understood  correctly  by 
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asking  questions,  paraphrasing,  and  repeating  the  message  back  to  the  speaker  (Daft,  2008;  Hughes  et  al., 
2012).  Active  listeners  also  pay  attention  to  speakers’  non-verbal  cues,  such  as  body  language,  tone, 
volume,  and  pace  of  speech,  to  attach  meaning  to  messages  that  are  received  (Daft,  2008;  Hughes  et  al., 
2012;  Thomas,  1974). 

Summary 

Additional  supervisory  behaviors,  such  as  coaching  and  mentoring  instructors  and  openly  and  honestly 
communicating  with  them,  enable  instructional  supervisors  to  be  more  effective  in  the  administrative  and 
classroom  roles  in  which  they  serve.  Supervisors’  ability  to  engage  in  these  and  other  effective 
instructional  supervisory  behaviors  is  further  enhanced  by  possessing  both  group  process  and  effective 
communication  skills. 


Instructional  Supervisor  Qualifications 

While  much  of  the  discussion  thus  far  has  focused  on  effective  instructional  supervisory  behaviors  and 
skills,  it  is  also  important  to  acknowledge  the  roles  education  and  experience  play  in  supervisory 
effectiveness.  In  this  section,  the  education  and  experience  requirements  for  the  position  of  instructional 
supervisor  are  discussed  to  develop  the  reader’s  understanding  of  how  these  qualifications  may  be 
considered  in  supervisor  selection,  training,  and  development  processes. 

Education 

To  optimize  their  effectiveness,  it  may  be  preferred  for  instructional  supervisors  to  hold  Master’s  or 
Doctoral  degrees  in  second  language  acquisition  or  instruction,  education  (e.g.,  adult  education  or 
andragogy),  or  in  other  relevant  fields.  Formal  graduate-level  education  in  these  fields  may  bolster 
supervisors’  ability  to  support  their  instructors’  efforts  in  the  classroom. 

Experience 

Along  with  education,  another  way  to  enhance  instructional  supervisory  effectiveness  is  to  hire  and 
promote  supervisors  who  possess  relevant  experience.  Research  indicates  that  experience  can  provide 
some  prediction  of  future  performance,  particularly  when  the  past  experience  closely  matches  the  current 
job  to  be  performed,  as  similarity  allows  for  knowledge  and  skill  transfer  (Dokko,  Wilk,  &  Rothbard, 
2009;  Quinones,  Ford,  &  Teachout,  1995).  In  the  SOF  IAT  context,  three  types  of  experience  may  be 
particularly  relevant:  (1)  administrative,  (2)  instructional,  and  (3)  military. 

A  dm  inistrative  Experience 

As  discussed  previously,  effective  instructional  supervisors  engage  in  a  variety  of  administrative 
behaviors.  If  supervisory  candidates  have  previous  job-related  or  other  relevant  experience  in 
administration  or  management,  then  these  candidates  may  be  better  equipped  to  perform  effectively  in 
instructional  supervisory  roles. 
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Instructional  Experience 

Given  the  roles  instructional  supervisors  play  in  developing,  coaching,  and  mentoring  instructors,  as  well 
as  their  classroom  responsibilities,  it  is  imperative  that  supervisors  have  prior  instructional  experience. 
Without  this  experience,  supervisors  may  not  be  considered  credible  by  the  instructors  they  supervise. 

Experience  in  a  Military  Environment 

Even  if  instructional  supervisors  have  relevant  graduate  degrees  and  both  administrative  and  instructional 
experience,  their  effectiveness  in  the  SOF  1AT  context  may  be  limited  if  they  do  not  have  experience  in  a 
military  environment  or  organization.  There  are  many  elements  of  the  SOF  1AT  context  that  make  it 
distinct  from  other  foreign  language  training  environments  (e.g.,  training  duration  and  contact  hours).  As 
such,  supervisory  candidates  who  possess  an  understanding  of  or  prior  experience  in  a  military 
environment  or  organization  may  be  preferred  over  candidates  who  do  not  possess  this  experience. 

Summary 

Formal  education  in  second  language  acquisition  or  instruction,  education,  or  other  relevant  fields  may 
optimize  supervisor  effectiveness  in  the  foreign  language  instructional  context.  Likewise,  prior 
administrative,  instructional,  and  military  experience  may  enhance  supervisory  effectiveness.  As  such, 
these  qualifications  must  be  taken  into  account  when  instructional  supervisors  are  selected  and  training 
and  development  interventions  for  instructional  supervisors  are  planned. 

Conclusion 

Instructional  supervision  is  focused  on  the  improvement  of  instruction  and  learning  outcomes  (Gebhard, 
1990;  Glanz  et  al.,  2007,  April;  Lunenberg,  1998;  Neagley  &  Evans,  1980;  Sergiovanni  &  Starratt,  1998). 
To  be  effective,  instructional  supervisors  must  perform  a  number  of  administrative  and  classroom 
supervisory  behaviors.  Their  effectiveness  in  these  two  behavioral  categories  may  be  further  enhanced  by 
engaging  in  additional  effective  behaviors,  such  as  coaching  and  mentoring  instructors,  and  by  possessing 
relevant  skills  and  qualifications,  such  as  effective  communication  skills,  graduate  education  in  relevant 
fields,  and  administrative,  instructional,  and  military  experience. 

In  Section  IV  (pp.  23-43),  these  behaviors,  skills,  and  qualifications  are  discussed  in  SOF  1AT  language 
schools  and  in  other  language  schools  external  to  the  SOF  1AT  community. 


9/28/2012 


©  SWA  Consulting  Inc.,  2012 
Technical  Report  [2012010632] 


Page  22 


SOFLO  Support  Project 


Best  Practices  for  Supervision  of  Instructors 


SECTION  IV:  SOF  IAT  CURRENT-STATE  AND  EXTERNAL  BENCHMARKING  STUDIES 
Purpose 

The  purpose  of  the  SOF  IAT  current-state  study  was  to  identify  instructional  supervisory  behaviors  and 
practices  currently  used  in  SOF  IAT  language  schools  by  interviewing  SOF  IAT  language  program 
administrators  and  language  program  vendors.  Likewise,  the  purpose  of  the  external  benchmarking  study 
was  to  identify  instructional  supervisory  behaviors  and  practices  currently  used  in  language  schools 
external  to  the  SOF  IAT  community  by  interviewing  administrators  of  these  schools.  Results  from  these 
studies  will  aid  SOF  IAT  leadership,  CLPMs,  and  instructional  supervisors  in  maintaining  and  further 
developing  effective  instructional  supervisory  behaviors  and  practices  in  SOF  IAT  language  schools. 

Research  Areas  and  Questions 

For  both  the  SOF  IAT  current-state  and  external  benchmarking  studies,  there  were  four  primary  research 
areas:  (1)  position  of  instructional  supervisor,  (2)  relationships  between  instructional  supervisors  and 
instructors,  (3)  strategies  and  methods  for  instructional  supervision,  and  (4)  classroom  observations  and 
assessments.  Research  questions  for  each  area  included: 

Position  of  Instructional  Supervisor 

•  How  many  instructional  supervisors  are  there  in  your  language  school  or  department? 

•  What  are  instructional  supervisors’  responsibilities? 

Relationships  between  Instructional  Supervisors  and  Instructors 

•  How  many  instructors  does  each  supervisor  supervise? 

•  What  are  some  of  the  challenges  supervisors  face  in  working  with  instructors? 

Strategies  and  Methods  for  Instructional  Supervision 

•  What  power  or  resources  do  supervisors  have  to  help  instructors  improve? 

•  What  strategies  are  used  to  motivate  instructors? 

Classroom  Observations  and  Assessments 

•  Who  conducts  classroom  observations? 

•  What  occurs  after  an  observation  is  conducted  or  an  instructor  is  assessed? 
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In  addition  to  these  four  research  areas,  two  additional  research  areas  were  examined  for  the  external 
benchmarking  study:  (1)  language  learning  environment,  and  (2)  language  learners.  These  research  areas 
were  examined  to  determine  how  similar  the  external  language  schools  were  to  the  SOF  IAT  language 
schools.  Research  questions  for  these  two  areas  included: 

Language  Learning  Environment 

•  What  is  the  course  duration  and  total  contact  hours? 

•  What  is  the  purpose  for  learning  the  language? 

Language  Learners 

•  What  is  the  gender  of  most  of  the  students  here? 

•  Do  most  of  the  students  have  previous  language  learning  experience  (i.e.,  high  school  or  college 
courses,  other  language  learning  programs)? 

Main  Findings 

After  interviewing  language  program  administrators  and  language  program  vendors  from  SOF  IAT 
schools  and  administrators  of  language  schools  external  to  the  SOF  IAT  community,  interview  responses 
were  content-coded  and  analyzed  to  examine  the  similarities  and  differences  between  instructional 
supervisory  behaviors  and  practices  used  in  SOF  IAT  language  schools  and  language  schools  external  to 
the  SOF  IAT  community.  The  main  findings  resulting  from  the  SOF  IAT  current-state  study  are: 

Position  of  Instructional  Supervisor 

•  Consistent  responsibilities  of  instructional  supervisors  across  all  SOF  IAT  language  schools 
include  training,  mentoring,  or  orienting  instructors,  monitoring  instructor  performance, 
providing  constructive  feedback,  and  providing  resources  to  instructors. 

•  Interpersonal,  motivational,  and  communication  skills  were  mentioned  by  SOF  IAT  interviewees 
as  important  for  effective  instructional  supervision.  In  addition,  content  knowledge  and  English 
proficiency  were  also  described  as  important  for  effective  supervision.  According  to 
interviewees,  being  closed-minded,  overly  critical,  or  lacking  interpersonal  skills  can  hinder 
instructional  supervisors’  ability  to  work  effectively  with  instructors. 

•  All  SOF  IAT  participants  reported  that  instructional  experience  is  a  required  or  preferred 
qualification  for  instructional  supervisors;  however,  the  reported  minimum  years  of  instructional 
experience  for  SOF  IAT  instructional  supervisors  ranged  from  0-5.  Previous  instructional 
experience  in  a  military  environment  or  organization  is  also  preferred. 

Relationships  between  Instructional  Supervisors  and  Instructors 

•  The  number  of  instructional  supervisors  in  each  SOF  IAT  language  school  is  similar  across  all 
components  interviewed,  but  there  is  a  wide  range  in  the  number  of  instructors  per  supervisor. 
The  reported  range  of  instructors  per  supervisor  was  as  low  as  5-10  and  as  high  as  35. 
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•  Common  challenges  SOF  1AT  instructional  supervisors  encounter  when  working  with  instructors 
include  instructors’  unwillingness  to  learn  or  cooperate  and  their  lack  of  cultural  understanding. 

Strategies  and  Methods  for  Instructional  Supervision 

•  In  SOF  IAT  language  schools,  instructional  supervisors  use  current  literature,  additional  training, 
discussions  with  instructors  and  attendance  at  professional  conferences  to  stay  informed  of 
current  best  practices  for  instruction. 

•  Instructional  supervisors  in  SOF  IAT  can  recommend  or  direct  instructors  to  resources,  such  as 
online  resources,  computer  software,  and  professional  development  opportunities.  Most 
supervisors  interviewed  indicated  they  know  of  these  resources  either  through  personal 
experience  or  their  own  research. 

•  Instructional  supervisors  can  also  introduce  or  encourage  the  use  of  particular  models  of 
instruction  in  the  classroom.  In  the  SOF  IAT  context,  the  models  promoted  are  primarily 
communicative,  task-based,  and  learner-centered. 

•  Within  SOF  IAT  language  schools,  instructional  supervisors  motivate  instructors  by  providing 
them  with  professional  development  opportunities  and  performance  feedback  and  by  showing 
instructors  their  appreciation  for  a  job  well  done. 

Classroom  Observations  and  Assessments 

•  SOF  IAT  participants  stated  that  instructional  supervisors  observe  each  instructor  at  least  twice 
per  course,  with  some  reporting  four  or  more  visits  during  each  course.  Most  participants 
indicated  that  frequency  varies  depending  on  instructor  tenure;  less-tenured  instructors  are 
observed  more  frequently  than  longer-tenured  instructors. 

•  Instructional  supervisors  in  all  SOF  IAT  language  schools  observe  low-performing  instructors 
more  frequently  than  high-performing  instructors.  This  also  varies  as  a  function  of  instructor 
tenure. 

•  All  SOF  IAT  participants  stated  that  the  classroom  observations  are  usually  scheduled  (or 
announced)  and  that  instructors  are  forewarned  of  supervisors’  attendance. 

•  The  length  of  classroom  observations  varies  across  SOF  IAT  language  schools.  The  average 
length  of  an  observation  is  one  hour. 

•  Performance  standards  used  as  metrics  during  observations  at  SOF  IAT  language  schools  are 
most  frequently  derived  from  instructors’  previous  performance  evaluations. 

•  After  classroom  observations  are  conducted,  notes  or  scores  are  reviewed  with  instructors,  and 
instructors  have  an  opportunity  to  dispute  or  explain  anything  their  supervisors  observed. 
Following  that,  some  participants  mentioned  that  improvement  plans  are  collaboratively 
developed  by  instructors  and  supervisors,  and  others  mentioned  creating  reports  covering  the 
observations  and  their  outcomes. 
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Method 


Participant  Identification 

To  conduct  interviews  for  both  the  SOF  IAT  current-state  and  external  benchmarking  studies,  it  was  first 
necessary  to  identify  participants  to  be  contacted. 

SOF  IAT  Current-State  Study 

For  the  current-state  study,  SOF  IAT  language  program  administrators  (Table  1,  p.  28)  and  current  SOF 
IAT  language  program  vendors  (Table  2,  p.  28)  for  the  following  five  SOF  IAT  language  schools  were 
identified: 

•  Air  Force  Special  Operations  Command  (AFSOC) 

•  Naval  Special  Warfare  (WARCOM) 

•  SWCS  Basic  Language  Course  (BLC) 

•  SWCS  Intermediate  Language  Course  (ILC) 

•  U.S.  Marine  Corps  Forces  Special  Operations  Command  (MARSOC) 

Across  the  five  SOF  IAT  language  schools,  1 1  language  program  administrators  and  12  current  language 
program  vendors  were  identified  to  request  their  participation  in  the  SOF  IAT  current-state  study. 

External  Benchmarking  Study 

For  the  external  benchmarking  study,  administrators  from  language  schools  external  to  the  SOF  IAT 
community  were  identified  to  participate  in  an  interview  regarding  instructional  supervisory  behaviors 
and  practices  currently  used  in  their  language  schools.  These  language  schools  were  selected  for 
participation  based  on  a  number  of  relevant  criteria,  such  as  their  reputations  in  the  fields  of  second 
language  acquisition  and  education  and  their  similarity  to  the  SOF  IAT  context.  A  total  of  22  language 
schools  were  identified  (Table  3,  p.  29).  For  nine  of  these  language  schools,  multiple  points  of  contact 
(POCs)  were  identified.  Thus,  the  total  number  of  POCs  identified  to  request  their  participation  in  the 
external  benchmarking  study  was  35. 

Participant  Contact 

Participants  who  were  identified  for  both  the  current-state  and  external  benchmarking  studies  were 
initially  contacted  via  email  to  request  their  participation  in  an  hour-long  telephone  interview  conducted 
by  an  SWA  researcher.  The  email  detailed  the  intent  of  the  study  and  included  a  statement  regarding  how 
SWA  researchers  would  use  the  data  collected  during  the  interviews. 

A  formal  letter,  individually  addressed  to  each  participant,  was  attached  to  the  e-mail  each  participant 
received.  Slightly  different  versions  of  this  letter  were  created  for  individuals  in  each  of  the  three 
participant  groups:  (1)  SOF  IAT  language  program  administrators  (Appendix  A,  pp.  55-57),  (2)  SOF  IAT 
language  program  vendors  (Appendix  B,  pp.  58-60),  and  (3)  administrators  from  language  schools 
external  to  the  SOF  IAT  community  (Appendix  C,  pp.  61-63). 
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Each  letter  contained  a  uniform  resource  locator  (URL)  for  an  online  interest  survey.  By  completing  this 
interest  survey,  each  participant  would  indicate  his  or  her  interest  in  participating  in  the  study  and  general 
availability  for  the  interview.  Participants  who  did  not  respond  to  the  online  interest  survey  within 
several  business  days  were  contacted  again  via  email  to  request  their  participation. 

Interview  Protocol  Development  and  Participation 

Interview  Protocol  Development 

The  interview  protocols  for  both  the  current-state  (Appendix  D,  pp.  64-67)  and  external  benchmarking 
studies  (Appendix  E,  pp.  68-74)  were  developed  by  SWA  researchers  expressly  for  these  studies.  The 
protocols  were  based  on  a  systematic  review  of  theoretical,  empirical,  and  best  practices  literature  in  the 
fields  of  education,  psychology,  second  language  acquisition,  and  training  (Section  111,  pp.  11-22).  These 
reviews  allowed  SWA  researchers  to  identify  a  list  of  effective  instructional  supervisory  behaviors,  skills, 
and  qualifications;  from  this  list,  interview  questions  were  developed.  This  approach  to  interview 
question  creation  allowed  SWA  researchers  to  compare  instructional  supervisory  behaviors  and  practices 
currently  used  in  SOF  1AT  language  schools  to  practices  recommended  in  the  literature  on  this  topic,  as 
well  as  to  practices  currently  used  in  language  schools  external  to  the  SOF  1AT  community. 

Upon  completion,  the  interview  protocols  for  both  the  current-state  and  external  benchmarking  studies 
were  submitted  to  the  Deputy  Chief  of  FMD-J9  for  an  independent  review  of  the  studies’  exemption 
qualification.  Under  Title  32,  §219  of  the  Code  of  Federal  Regulations  from  the  Department  of  Defense 
Instruction  3216.02,  these  studies  qualified  as  exempt  because  the  information  interviewees  provided  is 
not  published  in  this  report  showing  how  specific  individuals  responded,  identities  are  protected,  and 
groupings  of  responses  by  external  organization  (for  the  external  benchmarking  study)  are  not  shown. 

For  the  current-state  and  external  benchmarking  studies,  the  interview  protocols  included  greetings, 
introductory  statements,  interview  questions,  and  closing  statements.  Though  the  interview  questions  for 
both  the  current-state  and  external  benchmarking  studies  were  largely  the  same,  the  interview  protocol  for 
the  external  benchmarking  study  included  two  additional  sets  of  questions  pertaining  to  the  language 
learning  environment  and  language  learners. 

Interview  Participation 

After  developing  the  interview  protocols  and  contacting  individuals  in  the  three  participant  groups, 
telephone  interviews  were  scheduled  with  interested  participants  in  accordance  with  their  availability. 

For  each  telephone  interview  that  was  conducted,  there  was  one  interviewer  and  one  scribe  from  SWA. 

A  total  of  22  interviews  were  conducted  across  the  three  participant  groups.  Thus,  of  the  58  individuals 
contacted,  37.93%  actually  participated  in  interviews  (Table  4,  p.  30).  By  participant  group,  the  highest 
rate  of  participation  was  in  the  SOF  1AT  language  program  administrators  group  (n  =  8;  72.72%). 
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Table  1.  Current-State  Study  Participation  by  SOF  IAT  Language  Program  Administrators  by  Component 


SOF  IAT  Component 

Number  Contacted 

Number  Interested 

Number  Interviewed 

AFSOC 

2 

2 

1 

MARSOC 

2 

2 

2 

SWCS  BLC 

41 

3 

3 

SWCS  ILC 

2 

1 

1 

WARCOM 

1 

1 

1 

Total 

11 

9 

8 

‘This  number  includes  two  SWCS  BLC  Department  Chairs.  The  third  SWCS  BLC  Department  Chair  was  contacted  to  participate,  but  her  email  was  returned. 


Table  2.  Current-State  Study  Participation  by  SOF  IAT  Language  Program  Vendors  by  Component 


SOF  IAT  Component 

Vendor  Name  ( Number  Contacted) 

Number 

Interested 

Number 

Interviewed 

AFSOC 

Defense  Language  Institute  ( 7 ) 

1 

1 

MARSOC 

Comprehensive  Language  Center,  Inc.  (7);  Defense  Language  Institute  (2) 

2 

2 

SWCS  BLC 

North  Carolina  State  University  (7);  Comprehensive  Language  Center,  Inc.  (2) 

1 

1 

SWCS  ILC 

Defense  Language  Institute  (2);  Diplomatic  Language  Services  (2) 

2 

2 

WARCOM 

Group  SSI  Inc.  (7) 

0 

0 

Total 

12 

6 

6 

9/28/2012 


©  SWA  Consulting  Inc.,  2012 
Technical  Report  [2012010632] 


Page  28 


SOFLO  Support  Project 


Best  Practices  for  Supervision  of  Instructors 


Table  3.  External  Benchmarking  Study  Participation  by  Administrators  of  Language  Schools  External  to  SOF  1AT 


External  Language  School 

Number  Contacted 

Number  Interested 

Number  Interviewed 

American  Council  on  the  Teaching  of  Foreign  Languages 

1 

0 

0 

Brigham  Y oung  University 

1 

0 

0 

Center  for  Applied  Linguistics 

2 

0 

0 

Concordia  Language  Villages 

2 

1 

1 

Defense  Language  Institute  Foreign  Language  Center 

3 

0 

0 

Defense  Language  Institute  Washington  Office 

1 

0 

0 

Flagship  National  Security  Education  Program  Headquarters 

1 

0 

0 

Florida  State  University 

1 

0 

0 

Foreign  Service  Institute 

2 

0 

0 

Language  Acquisition  Resource  Center,  San  Diego  State  University 

3 

3 

2 

Lauder  Institute,  University  of  Pennsylvania 

1 

0 

0 

Michigan  State  University 

1 

0 

0 

Middlebury  College 

2 

1 

1 

Monterey  Institute  of  International  Studies 

1 

0 

0 

National  Foreign  Language  Center 

2 

0 

0 

National  Security  Agency 

4 

0 

0 

Stanford  Language  Institute 

1 

1 

1 

United  States  Air  Force  Academy 

1 

1 

1 

United  States  Military  Academy  at  West  Point 

1 

1 

1 

University  of  Illinois  -  Urbana-Champaign 

2 

1 

1 

University  of  Montana 

1 

0 

0 

Washington  University 

1 

0 

0 

Total 

35 

9 

8 
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Table  4.  Interview  Interest  and  Participation  Rates  by  Participant  Group 


Participant  Group 

Number  Interested 

Number  Interviewed 

(Interest  Response  Rate) 

(Interview  Participation  Rate)1 

SOF  IAT  Language  Program  Administrators 

9  (75%) 

8  (72.72%) 

SOF  IAT  Language  Program  Vendors 

6  (50%) 

6  (50%) 

Administrators  at  Language  Schools  External  to  SOF  IAT 

9(25.71%) 

8  (22.86%) 

Overall 

24  (41.38%) 

22  (37.93%) 

'The  interview  participation  rates  by  participant  group  represent  the  percentage  of  the  total  number  of  participants  in  that  group  who  were  initially  contacted  who  actually 
participated  interviews.  For  example,  1 1  SOF  IAT  language  program  administrators  were  initially  contacted  to  participate  in  interviews  and  eight  (72.72%)  of  these  individuals 
actually  participated  in  interviews. 
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Results 

Results  of  the  interviews  conducted  for  both  the  SOF  1AT  current-state  and  external  benchmarking 
studies  are  presented  in  five  sections.  For  these  results,  the  responses  provided  by  SOF  1AT  language 
program  administrators  and  SOF  1AT  language  program  vendors  for  each  language  school  were 
combined;  likewise,  the  responses  provided  by  administrators  of  language  schools  external  to  the  SOF 
1AT  community  were  also  combined.  Interview  responses  from  SOF  1AT  language  program 
administrators  and  SOF  1AT  language  program  vendors  were  analyzed  together;  there  were  too  few 
responses  within  each  language  school  to  allow  for  separate  analyses  by  administrators  and  vendors 
within  each  language  school.  In  general,  results  are  presented  by  SOF  1AT  language  school  and  across  all 
language  schools  that  participated  in  the  external  benchmarking  study. 

Part  1  presents  information  provided  by  administrators  from  language  schools  external  to  the  SOF  1AT 
community.  This  section  focuses  on  language  learners  and  the  language  learning  environment  and 
contains  three  additional  sections:  (1)  purpose  for  learning  the  language,  (2)  course  duration  and 
frequency  of  class  meetings,  and  (3)  proficiency  assessments.  These  findings  are  presented  to  allow  the 
reader  to  judge  the  similarities  and  differences  between  external  language  schools  interviewed  for  the 
external  benchmarking  study  and  SOF  1AT  language  schools  interviewed  for  the  current-state  study.  The 
results  presented  in  Part  I  should  be  used  to  caveat  the  findings  presented  in  Parts  11-V,  particularly  when 
the  language  learners  and  language  learning  environments  of  the  external  language  schools  differed 
substantially  from  the  SOF  1AT  language  schools. 

Part  11  focuses  on  the  position  of  instructional  supervisor.  This  section  presents  results  from  both  the  SOF 
1AT  current-state  and  external  benchmarking  studies  and  contains  two  sections:  (1)  responsibilities,  and 
(2)  instructional  supervisory  qualifications  (i.e.,  language  proficiency  and  cultural  expertise,  content 
knowledge,  instructional  experience,  administrative  experience,  qualities  of  effective  instructional 
supervisors,  and  other  qualifications).  These  findings  are  presented  to  inform  the  reader  about 
supervisors’  responsibilities  in  SOF  1AT  language  schools  and  external  language  schools,  as  well  as  the 
qualifications  that  enhance  supervisors’  ability  to  effectively  perform  these  responsibilities. 

Part  111  examines  relationships  between  supervisors  and  instructors.  This  section  presents  results  from 
both  the  SOF  1AT  current-state  and  external  benchmarking  studies  and  contains  two  sections:  (1)  ratio  of 
instructors  to  supervisors,  and  (2)  challenges  in  working  with  instructors. 

Part  IV  describes  strategies  and  methods  for  instructional  supervision  used  in  both  SOF  1AT  language 
schools  and  language  schools  external  to  the  SOF  1AT  community.  This  section  contains  three  sections: 
(1)  professional  development  (for  both  instructional  supervisors  and  instructors),  (2)  models  of 
instruction,  and  (3)  instructor  motivation. 

Part  V  summarizes  information  about  classroom  observations  and  assessments,  such  as  who  conducts 
observations,  how  often  observations  are  conducted,  and  how  long  observations  are.  Observing  and 
assessing  foreign  language  instructors  in  their  classrooms  are  two  of  the  main  responsibilities 
instructional  supervisors  have  in  both  SOF  1AT  and  external  language  schools. 
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Part  I:  Language  Learners  and  Language  Learning  Environment 

When  asked  about  language  learners  and  the  language  learning  environment  in  their  schools, 
administrators  from  external  language  schools  described  learner  characteristics  and  learning  environments 
that  differed  somewhat  from  SOF  1AT  language  schools. 

•  Native  language  and  previous  language  experience.  In  the  areas  of  student  native  language  and 
previous  language  experience,  there  were  no  differences  between  participant  groups;  for  the 
majority  of  language  learners,  their  native  language  was  English  and  they  had  at  least  some  prior 
language  training,  such  as  high  school  courses. 

•  Gender.  The  gender  of  the  students  varied  across  participant  groups,  with  external  language 
schools  reporting  a  predominantly  female  population,  and  SOF  1AT  language  schools  reporting  a 
majority  male  population. 

•  Industry.  There  was  substantial  variability  in  the  responses  provided  by  external  language  school 
administrators  in  terms  of  the  industry  in  which  their  students  were  employed.  Students  in 
language  schools  external  to  the  SOF  1AT  community  were  employed  in  many  different 
industries,  including  education,  business,  and  medical  fields.  Further  results  should  be  interpreted 
with  this  difference  in  mind,  as  the  industries  in  which  these  students  were  employed  could 
impact  their  purposes  for  language  learning. 

Purpose  for  Language  Learning 

In  six  of  the  seven  external  language  schools,  administrators  reported  students  were  required  to  take  the 
courses,  either  by  their  academic  institutions  ( n  =  4),  the  military  (n  =  2),  or  business  employers  (n  =  1; 
Figure  3,  p.  32).  In  the  remaining  external  language  school,  administrators  indicated  students  were 
enrolled  in  language  learning  of  their  own  volition.  Some  examples  of  these  voluntary  learners  were 
those  who  took  courses  for  their  own  self-improvement  purposes  and  those  who  were  training  to  fulfill 
elective  academic  credits. 

Figure  3.  Purpose  for  Language  Learning  in  External  Language  Schools 


Required  by  Required  by  Required  by  Training  not 
business  military  academic  required 

institution 
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Administrators  from  external  language  schools  were  asked  to  provide  an  assessment  of  their  students’ 
motivation  to  learn  their  languages.  Although  one  might  assume  learners  who  are  required  to  take 
language  courses  may  be  less  motivated  than  voluntary  learners,  reports  from  external  administrators 
revealed  that  was  not  the  case.  According  to  these  administrators,  motivation  was  high  for  students  in 
these  language  schools,  regardless  of  whether  or  not  the  courses  were  required.  Of  those  who  provided 
responses,  83%  (//  =  5)  indicated  students  in  their  language  schools  were  highly  motivated  to  learn  their 
languages.  Motivation  levels  varied  based  on  language,  course  level,  and  industry  of  employment. 

Course  Duration  and  Frequency  of  Class  Meetings 

Language  schools  external  to  the  government  offered  courses  in  many  of  the  languages  currently  taught 
within  the  SOF  1AT  community  (Figure  4,  p.  33).  The  number  of  languages  offered  varied  greatly 
between  these  schools;  administrators  stated  their  respective  language  schools  had  between  4-40  different 
language  offerings,  with  most  schools  offering  about  eight  languages.  Across  all  external  language 
schools  interviewed,  Arabic  (13%)  and  Russian  (11%)  were  the  most  commonly  reported  language 
offerings,  followed  by  Chinese  (9%),  French  (9%),  Portuguese  (9%),  and  Spanish  (9%). 


Figure  4.  Languages  Offered  in  Language  Schools  External  to  SOF 

Korean  Swedish 


For  many  of  the  external  language  schools  interviewed,  the  duration  of  training  varied  in  accordance  with 
the  languages  being  taught.  In  the  SOF  1AT  community,  languages  are  divided  into  categories  (i.e.,  Cat  1, 
II,  III,  and  IV)  based  on  difficulty  for  a  native  English  speaker.  The  majority  of  the  external  schools 
interviewed  offered  an  approximately  even  split  between  Cat  I/II  and  Cat  III/IV  languages. 

Certain  external  language  schools  also  offered  iso-immersion  programs,  and  these  immersion  programs 
ranged  between  one  weekend  and  four  weeks  in  length,  based  on  language  difficulty  and  student  need. 
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Other  external  language  schools  followed  a  more  traditional  academic  schedule,  with  courses  lasting  one 
semester  and  meeting  either  three  or  five  days  a  week  for  one  to  a  few  hours  per  class  meeting. 

Regardless  of  the  variation,  seven  of  the  eight  external  language  program  administrators  stated  that  course 
duration  and  frequency  of  class  meetings  differed  based  on  language  difficulty  and  course  level. 

Proficiency  Assessments 

The  timing  of  proficiency  estimations  relative  to  course  progress  varied  greatly  across  language  schools 
external  to  SOF  1AT.  Six  of  the  seven  external  schools  who  provided  a  response  administered  some  form 
of  proficiency  test  at  the  end  of  language  training.  However,  only  three  of  the  schools  did  proficiency 
assessments  at  the  beginning,  in  the  middle,  and  at  the  end  of  training.  The  three  remaining  external 
language  schools  assessed  student  proficiency  twice  per  course  (i.e.,  one  assessment  at  the  end  of  training 
and  one  at  another  time  during  the  course). 

In  general,  external  language  schools  interviewed  for  the  benchmarking  study  did  not  necessarily  have 
specific  targeted  proficiency  levels  students  were  expected  to  attain  by  the  end  of  training.  However, 
incoming  proficiency  levels  of  the  students  often  affected  when  assessments  were  administered.  External 
administrators  who  stated  their  schools  predominantly  enrolled  students  at  beginner  levels  of  proficiency 
did  not  administer  assessments  before  training  began.  In  contrast,  schools  with  a  range  of  incoming 
student  proficiency  levels  reported  administering  assessments  at  all  three  time  points  (i.e.,  before,  during, 
and  at  the  end  of  language  training). 

Part  II:  Position  of  Instructional  Supervisor 

Responsibilities 

In  both  SOF  1AT  and  external  language  schools,  instructional  supervisors  have  a  diverse  set  of 
responsibilities.  Participants  in  both  the  current-state  and  external  benchmarking  studies  were  asked 
about  instructional  supervisors’  responsibilities.  For  language  schools  external  to  the  SOF  1AT 
community,  all  interviewees  indicated  that  supervisors  in  their  organizations  monitor  instructor 
performance,  provide  constructive  feedback  to  instructors,  provide  resources  to  instructors,  and  develop 
curriculum  (Figure  5,  p.  35).  This  is  similar  to  SOF  1AT  language  schools  in  that  all  schools’  supervisors 
monitor  instructor  performance,  provide  constructive  feedback  to  instructors,  and  provide  resources  to 
instructors  (Table  5,  p.  35).  However,  differences  lie  in  that  SOF  1AT  respondents  unanimously  indicated 
that  their  instructional  supervisors  also  train,  mentor,  and  orient  instructors,  but  they  do  not  all  develop 
curriculum.  The  other  responsibilities  of  instructional  supervisors  vary  across  organizations  interviewed 
for  both  the  current-state  and  benchmarking  studies. 
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Figure  5.  Responsibilities  of  Instructional  Supervisors  in  External  Language  Schools 


Interview/hire  instructors 


Train,  mentor,  orient  instructors 
Monitor  instructor  performance 
Provide  incentives/rewards 


Provide  constructive  feedback 


Provide  resources 


Schedule  courses 


Select  or  specify  curriculum 
Develop  curriculum 


50% 


100% 


a  Yes 
■  No 


Table  5.  Responsibilities  of  Instructional  Supervisors  by  SOF  1AT  Language  School 


Responsibilities 

AFSOC 

MARSOC 

swcs 

BLC 

SWCS 

ILC 

WARCOM 

Interview/hire  instructors 

Yes 

Yes 

Yes 

Split 

No 

Train,  mentor,  orient  instructors 

Yes 

Yes 

Yes 

Yes 

Yes 

Monitor  instructor  performance 

Yes 

Yes 

Yes 

Yes 

Yes 

Provide  incentives/rewards 

Split 

No 

Yes 

Split 

No 

Provide  constructive  feedback 

Yes 

Yes 

Yes 

Yes 

Yes 

Provide  resources 

Yes 

Yes 

Yes 

Yes 

Yes 

Schedule  courses 

Split 

No 

Split 

No 

N/A 

Select  or  specify  curriculum 

Yes 

No 

Yes 

Split 

No 

Develop  curriculum 

Split 

Yes 

No 

No 

No 

Lead  meetings 

Yes 

Yes 

Yes 

Split 

N/A 

Note.  Split  responses  indicate  disagreement  between  interviewees  from  that  language  school.  N/A  =  No  answer. 


There  were  other  key  differences  between  participating  language  schools,  aside  from  instructional 
supervisors’  responsibilities.  For  example,  instructional  supervisory  positions  were  structured  differently 
in  the  language  schools  interviewed,  in  terms  of  organizational  level  and  reporting  relationships. 
Instructional  supervisors  in  SOF  IAT  schools  report  to  government  superiors,  contract  vendor  superiors, 
or  both,  whereas  in  external  language  schools,  supervisors  reported  to  anyone  from  a  program  director  to 
a  school  dean.  Some  of  the  differences  in  responsibilities  across  SOF  IAT  and  external  language  schools 
could  be  the  result  of  the  placement  of  instructional  supervisory  positions  within  their  respective 
organizational  structures.  Differences  could  also  be  a  result  of  differences  in  the  relationship  between  the 
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component  and  its  vendors.  Some  vendors  require  their  instructors  be  managed  by  only  vendor 
instructional  supervisors,  while  vendors  at  other  components  may  work  more  collaboratively  with 
government  instructional  supervisors,  and  the  responsibilities  will  be  more  balanced  between  vendor  and 
government  instructional  supervisors. 

Instructional  Supervisory  Qualifications 

Language  proficiency  and  cultural  expertise.  Participants  in  both  the  SOF  1AT  current-state  and  external 
benchmarking  studies  were  asked  about  the  importance  of  language  proficiency,  cultural  expertise,  and 
English  proficiency  to  instructional  supervisory  effectiveness  (Table  6,  p.  36).  Overall,  though  several 
participants  did  indicate  that  language  proficiency  is  important  to  instructional  supervisory  effectiveness, 
the  general  consensus  is  that  cultural  expertise  is  more  important  than  language  proficiency,  particularly 
when  supervisors  manage  instructors  in  languages  in  which  they  are  not  proficient.  If  a  supervisor 
respects  and  understands  instructors  and  their  ethnic  and  cultural  backgrounds,  then  the  ability  to  speak 
their  language  is  less  crucial  to  supervisory  effectiveness.  Further,  rather  than  having  supervisors  who  are 
proficient  in  multiple  languages,  most  participants  indicated  they  would  prefer  to  have  supervisors  who 
can  effectively  communicate  with  their  colleagues  in  English. 


Table  6.  Importance  of  Language  and  Cultural  Proficiency  to  Supervisory  Effectiveness 


Is  it  important  to 
have  supervisors  who 
are  proficient  in: 


AFSOC 

MARSOC 

SWCS  BLC 

SWCS  ILC 

WARCOM 

External 

Mixed 

No 

No 

Yes 

No 

Mixed 

Yes 

Yes 

Yes 

Yes 

No 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Note.  A  response  of  Mixed  indicates  there  was  no  clear  consensus  among  participants  from  that  language  school. 


Content  knowledge.  Participants  in  both  the  SOF  1AT  current-state  and  external  benchmarking  studies 
indicated  that  content  knowledge  of  second  language  acquisition  or  education  is  important  to  instructional 
supervisory  effectiveness. 


Instructional  experience.  Participants  from  all  language  schools  (i.e.,  SOF  1AT  and  external  language 
schools)  indicated  their  instructional  supervisors  were  once  instructors.  In  the  SOF  IAT  language 
schools,  in  particular,  the  requirements  for  minimum  years  of  instructional  experience  differ  from  one 
language  school  to  another  (Figure  6,  p.  37). 
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Figure  6.  Minimum  Years  of  Instructional  Experience  for  Instructional  Supervisors 


Administrative  experience.  In  addition  to  prior  instructional  experience,  participants  were  also  asked 
about  the  typical  background  of  instructional  supervisors  in  their  language  schools.  Of  the  external 
language  school  administrators  who  responded  to  this  question,  100%  (n  =  3)  indicated  their  supervisors 
also  had  prior  experience  as  administrators.  This  is  almost  identical  to  the  responses  provided  by  SOF 
IAT  participants,  86%  (n  =  7)  of  whom  also  said  their  instructional  supervisors  were  once  administrators. 

Qualities  of  effective  instructional  supervisors.  Participants  were  asked  to  reflect  on  examples  of  effective 
and  ineffective  instructional  supervision.  In  describing  these  examples,  participants  highlighted  a 
plethora  of  effective  and  ineffective  supervisory  qualities  and  skills.  Specific  to  SOF  IAT  participants, 
leadership  skills  and  effective  communication  skills  were  listed  as  highly  important  to  supervisory 
effectiveness.  SOF  IAT  participants  also  reported  that  motivational  skills  and  the  accessibility  of 
supervisors  are  important,  while  none  of  the  external  language  schools  mentioned  them  as  important. 

For  both  the  current  state  and  benchmarking  studies,  some  of  the  most  frequently  mentioned  skills  for 
effective  supervision  included  mentorship  abilities,  interpersonal  skills,  cultural  understanding,  and 
pedagogical  skill,  while  being  closed-minded,  overly  critical,  or  lacking  interpersonal  skills  were 
commonly  described  in  examples  of  ineffective  instructional  supervision. 

Requirements.  When  asked  about  degree  or  educational  requirements  for  instructional  supervisors,  there 
was  very  little  consistency  in  responses.  Of  the  12  responses  from  SOF  IAT  current-state  participants, 
Master’s  degree  was  the  most  frequently  mentioned  qualification  ( n  =  6),  followed  by  teaching 
experience  and  a  background  in  education.  The  next  most  common  responses  from  SOF  IAT  participants 
were  management  or  business  experience  (n  =  3)  and  experience  in  the  areas  of  languages,  culture, 
literature,  or  linguistics  (n  =  3).  Surprisingly,  previous  experience  with  SOCOM  or  other  military  was 
noted  only  once  as  a  requirement  for  instructional  supervisors  in  SOF  IAT  language  schools. 

The  qualifications  for  instructional  supervisors  in  external  language  schools  differed  from  those  in  SOF 
IAT  language  schools;  a  minimum  Doctoral  degree  was  mentioned  in  two-thirds  of  responses  in  the 
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benchmarking  study  ( n  =  4).  The  next  most  common  requirement  was  target  language  proficiency,  with 
50%  (/?  =  3)  of  interviewees  reporting  that  instructional  supervisors  must  be  native  speakers  of,  or  have 
very  high  proficiency  in,  their  target  language. 

Part  III:  Relationships  between  Instructional  Supervisors  and  Instructors 

Ratio  of  Instructors  to  Supervisors 

While  interview  participants  from  SOF  1AT  language  schools  reported  having  between  1-5  instructional 
supervisors  for  each  language  school,  there  is  a  wide  range  for  the  number  of  instructors  per  supervisor 
(Table  7,  p.  38).  MARSOC  reported  the  smallest  ratio,  with  each  instructional  supervisor  responsible  for 
5-10  instructors.  The  most  extreme  case  was  reported  by  SWCS  1LC,  where  each  instructional  supervisor 
is  responsible  for  approximately  35  instructors.  SOF  1AT  interviewees  reported  spending  about  30 
minutes  one-on-one  with  each  instructor  per  week.  The  range  of  responses  from  external  language 
schools  was  too  large  to  be  comparable  to  SOF  1AT  language  schools. 

Table  7.  Number  of  SOF  1AT  Instructional  Supervisors  and  Instructors  per  Supervisor 


Number  of  Supervisors 

Number  of  Instructors 
per  Supervisor 

AFSOC 

1 

10 

MARSOC 

3 

5-10 

SWCS  BLC 

3 

10-28 

SWCS  ILC 

5 

up  to  35 

WARCOM 

3 

17 

According  to  all  participants  from  SOF  1AT  and  external  language  schools,  instructional  supervisors 
predominantly  supervise  instructors  of  only  one  type  (e.g.,  government  service  [GS]  or  contract,  full-time 
or  part-time,  Master’s  degree  or  Doctoral  degree).  In  SOF  1AT  language  schools,  91%  (n  =  10)  reported 
their  supervisors  are  responsible  for  only  one  type  of  instructor  (e.g.,  GS).  This  is  not  necessarily  viewed 
as  positive,  since  within  one  component,  there  is  usually  a  mix  of  GS  and  contract  instructors.  Since  each 
type  must  have  their  own  supervisor,  inconsistencies  can  arise.  One  SOF  1AT  interviewee  said,  “Ideally, 
we  would  have  just  one  type  of  employee. .  .and  that  way  they  can  be  the  supervisor  for  every  type  of 
instructor  to  ensure  all  instructors  have  the  same  type  of  training.” 

Challenges  in  Working  with  Instructors 

Instructional  supervisors  may  encounter  a  number  of  challenges  while  working  with  foreign  language 
instructors.  In  general,  interviewees  from  both  participant  groups  (i.e.,  SOF  1AT  schools  and  external 
language  schools)  mentioned  the  same  challenges  faced  by  their  instructional  supervisors  while  working 
with  instructors  (Figure  7,  p.  39). 

The  challenge  described  most  often  by  SOF  1AT  participants  was  working  with  instructors  who  are 
uncooperative  or  unwilling  to  learn  or  adapt  to  new  ways  of  teaching  their  classes.  An  example  response 
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from  a  SOF  IAT  interviewee  says,  “the  [vendor]  instructors  come  in  thinking  they  know  everything 
already.”  Also,  instructors’  inexperience  or  lack  of  familiarity  with  teaching  in  a  military  environment 
was  mentioned  as  a  challenge  by  23%  ( n  =  3)  of  the  SOF  IAT  participants.  This  can  be  due  to  the  fact 
that  vendors  are  used  to  supply  instructors,  and  therefore  they  may  have  never  taught  in  a  military  training 
environment.  One  respondent  said,  “They  don’t  know  how  the  government  system  works,”  and  another 
commented  that  it  takes  supervisors  “a  good  12  months  to  help  that  [instructor]  adapt  to  the 
environment.” 

Both  SOF  IAT  participants  and  administrators  from  external  language  schools  frequently  listed  lack  of 
cultural  understanding  as  a  challenge  faced  by  their  instructional  supervisors.  Instructors’  lack  of  cultural 
understanding  can  affect  their  supervisors’  ability  to  provide  help  or  advice  to  their  instructors.  It  could 
also  be  the  cause  of  conflict  between  instructors.  One  interviewee  described  the  difficulty  of  supervising 
such  an  instructor:  “If  their  culture,  particularly  their  professional  culture  of  origin,  is  very  different  from 
Western  culture,  it’s  a  challenge  to  get  used  to  teaching  and  working  in  a  Western  culture.” 

Figure  7.  Challenges  Faced  by  Instructional  Supervisors  while  Working  with  Instructors 


Unwillingness  to  learn  or  cooperate 
Lack  of  cultural  understanding 
Lack  of  education  or  teaching  experience 

Poor  attitude 

Unfamiliarity  with  language  learning  context 
Poor  time  management 

0%  10%  20%  30%  40%  50% 

■  SOF  IAT  (n=  12)  External  schools  (n=  5) 

Part  IV:  Strategies  and  Methods  for  Instructional  Supervision 

Professional  development.  To  be  effective,  instructional  supervisors  must  engage  in  their  own 
professional  development.  Supervisors  must  stay  up-to-date  on  the  latest  research  findings  on  and  current 
best  practices  for  second  language  acquisition.  If  they  do,  then  they  will  be  better  able  to  support 
instructors’  efforts  in  the  classroom  and  evaluate  instructors’  performance.  Examples  of  professional 
development  activities  engaged  in  by  instructional  supervisors  in  both  SOF  IAT  and  external  language 
schools  included  reading  current  research  and  best  practices  literature,  receiving  additional  training,  and 
attending  professional  conferences. 

Effective  instructional  supervisors  also  provide  opportunities  for  instructors’  professional  development. 
Across  both  SOF  IAT  and  external  language  schools,  instructional  supervisors  suggested  or  provided 
instructors  with  supplemental  resources  and  professional  development  opportunities.  These  resources  and 
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opportunities  are  often  identified  by  supervisors  through  their  research  and  personal  experience.  In  SOF 
1AT  language  schools,  some  participants  stated  that  resources  are  occasionally  requested  by  their 
language  schools  or  are  recommended  by  the  government. 

Models  of  instruction.  Interestingly,  the  models  of  instruction  supervisors  encourage  their  instructors  to 
use  in  the  classroom  do  not  differ  between  SOF  1AT  language  schools  and  external  language  schools. 

The  most  commonly  mentioned  models  are  communicative,  task-based,  and  learner-centered. 

Instructor  motivation.  One  area  in  which  SOF  1AT  schools  and  external  language  schools  operate 
differently  is  instructor  motivation.  Due  to  the  traditional  academic  nature  of  the  external  language 
schools,  the  need  for  supervisors  to  motivate  their  instructors  is  not  the  same  as  in  SOF  1AT  language 
schools.  Only  two  external  language  school  administrators  indicated  instructional  supervisors  in  their 
schools  play  a  role  in  instructor  motivation. 

Within  the  SOF  1AT  language  schools,  many  strategies  or  methods  for  motivating  instructors  were 
described.  For  example,  supervisors  can  motivate  their  instructors  by  providing  professional  development 
opportunities  or  offering  to  send  them  to  training  events,  such  as  courses  and  workshops.  They  also 
motivate  instructors  by  providing  them  with  feedback  and  communicating  their  appreciation  for  a  job  well 
done.  Several  of  the  SOF  1AT  participants  indicated  that  instructors  in  their  language  schools  were 
intrinsically  motivated  by  students’  drive  to  learn  the  languages  to  which  they  have  been  assigned. 

However,  instructor  motivation  is  one  area  in  which  there  may  be  contract  issues.  For  GS  instructors, 
training  can  be  used  as  a  motivator  for  instructors.  But  one  SOF  1AT  interviewee  said,  “depending  on  the 
contractor,  and  frankly  the  way  the  contract  is  written,  the  government  cannot  train  the  contractors.” 
Similarly,  another  interviewee  spoke  about  the  professional  development  opportunities  for  contractor 
instructors:  “There  is  a  lot  more  that  could  be  done,  but  the  way  their  contracts  are  set  up,  they  don’t  get 
paid  for  them.”  This  one  difference  prohibits  SOF  1AT  instructional  supervisors  from  using  a  potentially 
powerful  motivator  with  some  instructors  (i.e.,  professional  development). 

Part  V:  Classroom  Observations  and  Assessments 

As  noted  previously,  participants  from  all  of  the  SOF  1AT  language  schools  indicated  instructional 
supervisors  are  responsible  for  monitoring  instructors’  performance.  One  way  in  which  instructional 
supervisors  accomplish  this  is  by  conducting  classroom  observations  and  providing  performance 
evaluations  and  feedback  to  instructors  after  observations  are  conducted.  As  is  reflected  in  Table  8  (p. 

41),  these  practices  may  differ  markedly  between  SOF  1AT  language  schools.  Due  to  the  wide  variety  of 
external  language  schools  interviewed,  those  practices  also  differed  greatly,  to  the  point  where 
presentation  of  results  is  not  tenable. 

The  duration  of  language  training  differs  across  participating  organizations,  so  the  frequency  with  which 
supervisors  made  classroom  observations  also  varied.  Most  participants  stated  that  supervisors  in  their 
language  schools  visited  each  instructor  at  least  once  during  the  course,  with  some  reporting  four  or  more 
visits  to  each  instructor’s  classroom  during  each  course.  Most  of  the  participants  indicated  that  the 
frequency  of  observation  varies  significantly  depending  on  instructor  tenure.  Once  instructors  have  gone 
several  years  with  good  or  satisfactory  reviews,  it  is  less  and  less  likely  they  will  be  observed  and 
assessed  with  the  same  frequency  as  newer  instructors.  Similarly,  in  all  language  schools  interviewed, 
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lower-performing  instructors  were  observed  and  assessed  more  often  than  their  higher-performing 
colleagues.  If  an  instructor  is  not  performing  well,  supervisors  in  both  SOF  1AT  and  external  language 
schools  tend  to  devote  more  time  to  observe  and  assist  that  instructor. 

Best  practices  literature  on  whether  classroom  observations  should  be  announced  or  unannounced  is 
mixed.  Of  the  language  schools  that  participated  in  both  the  SOF  1AT  current-state  and  external 
benchmarking  studies,  most  reported  that  they  do,  generally,  give  advance  warning  to  instructors  prior  to 
conducting  classroom  observations. 

Table  8.  Classroom  Observations  and  Assessments 


Practices 

AFSOC 

MARSOC 

SWCS  BLC 

SWCS  ILC 

WARCOM 

Who  conducts 
observations? 

Supervisors 

Supervisors 

and 

instructors 

Supervisors 

(Department 

Chairs) 

Supervisors 

Instructors 

How  often  are  they 

2  times  per 

2  times  per 

2  times  per 

4+  times  per 

3  times  per 

conducted? 

course 

course 

course 

course 

course 

Are  they  announced, 
unannounced,  or  both? 

Announced 

Both 

Both 

Announced 

Announced 

How  long  are  they? 

Part  of  a 
class  session 

Full  class 
session 

One  hour 

One  hour 

Full  class 
session 

Are  high-performing 
instructors  observed  as 

No 

No 
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Note.  The  frequency  of  classroom  observations  varies  for  WARCOM  based  on  instructor  tenure.  The  figure  represented  here  is 
for  an  instructor’s  first  year  of  teaching. 


Other  characteristics  of  classroom  observations  were  discussed  in  interviews,  including  the  length  of  a 
single  observation.  Time  spent  in  the  classroom  can  vary  from  one  observation  to  another,  even  within 
the  same  SOF  IAT  language  school.  Many  times,  this  is  based  on  instructor  tenure;  those  who  have  been 
teaching  within  the  SOF  IAT  community  longer  are  observed  less  frequently  and  their  observations  are 
not  as  long  as  less-tenured  instructors. 

The  process  of  classroom  observations  is  inconsistent  across  SOF  IAT  language  schools,  although  the 
main  elements  are  the  same.  The  performance  standards  used  to  evaluate  instructors  most  often  come 
from  instructors’  previous  performance  reviews,  both  in  external  language  schools  (60%)  and  in  SOF  IAT 
language  schools  (45%).  Also,  there  are  many  standard  steps  in  a  classroom  observation  and  assessment 
process,  derived  from  literature  on  the  topic.  Figure  8  (p.  43)  illustrates  the  possible  steps  involved  in  a 
classroom  observation  process,  and  bold  text  indicates  which  of  the  organizations  interviewed  utilize  each 
step.  Instructional  supervisors  in  all  SOF  IAT  language  schools  meet  with  the  observed  instructor  at 
some  point  after  the  assessment  has  taken  place.  In  that  meeting,  instructors  have  the  opportunity  to 
dispute  or  explain  anything  the  supervisor  noted.  Afterwards,  some  participants  reported  supervisors  will 
help  instructors  develop  individual  improvement  plans,  including  the  timing  of  the  next  observations 
conducted  by  their  supervisors.  Also,  after  these  post-observation  meetings  with  instructors,  some 
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supervisors  will  write  reports  describing  the  findings  from  their  observations  and  evaluations.  Some  of 
the  differences  in  the  observation  processes  across  components  may  be  a  result  of  the  relationship  with 
that  component’s  vendor(s).  One  interviewee  comment  illustrates  how  the  contract  instructors  are 
affected  by  this  challenge,  saying  that  after  a  classroom  observation,  “the  instructor  will  come  by  and  ask 
what  they  did  well  or  poorly,  and  [the  instructional  supervisor]  says  that  he  can’t  tell  them  directly,  but 
that  he. . .  will  give  a  report  to  their  manager  and  the  manager  gets  to  decide  what  to  share  with  them.” 
Therefore,  even  if  a  government  instructional  supervisor  can  and  does  observe  contract  instructors  in  their 
classrooms,  the  results  of  these  observations  must  be  given  to  managers  on  the  contract  side,  and  there  are 
no  guarantees  contract  instructors  will  receive  the  assessment  feedback. 

To  mitigate  the  challenges  of  government/vendor  relationships  faced  by  instructional  supervisors,  SOF 
1AT  interviewees  reported  that  instructional  supervisors  pursue  whatever  avenues  currently  available  to 
ensure  trainees  receive  the  best  possible  instruction.  “When  [supervisor]  saw  a  problem  on  the 
observation,  talked  to  [vendor]  instead  of  to  instructor,  and  they  take  care  of  it.”  This  statement  is  an 
example  of  instructional  supervisors  doing  what  they  can  to  pass  information  along,  despite  not  being 
able  to  directly  manage  contract  instructors. 
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Figure  8.  SOF  IAT  Language  School  Participation  in  Classroom  Observation  and  Assessment  Stages 
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SECTION  V:  ANALYSIS  OF  GAPS  BETWEEN  CURRENT  AND  DESIRED  STATES 
Purpose 

The  purpose  of  this  section  is  to  identify  gaps  between  current  and  desired  states  of  instructional 
supervision  in  SOF  IAT  language  schools.  These  gaps  were  identified  by  comparing  instructional 
supervisory  behaviors,  skills,  and  qualifications  currently  used  in  SOF  IAT  language  schools  to  those 
identified  in  the  theoretical,  empirical,  and  best  practices  literature  on  this  topic.  Gaps  were  also 
identified  by  comparing  the  current  state  of  instructional  supervision  in  SOF  IAT  language  schools  to 
instructional  supervisory  behaviors  and  practices  used  in  language  schools  external  to  the  SOF  IAT 
community.  Recommendations  for  addressing  these  gaps  are  made  to  assist  SOF  leadership,  CLPMs,  and 
instructional  supervisors  in  maintaining  and  further  developing  effective  instructional  supervisory 
behaviors  and  practices  in  SOF  IAT  language  schools. 

Research  Questions 

•  What  gaps  exist  between  instructional  supervisory  behaviors  and  practices  used  in  SOF  IAT 
language  schools,  as  compared  to  the  theoretical,  empirical,  and  best  practices  literature  on  this 
topic? 

•  What  gaps  exist  between  instructional  supervisory  behaviors  and  practices  currently  used  in  SOF 
IAT  language  schools,  as  compared  to  instructional  supervisory  behaviors  and  practices  used  in 
language  schools  external  to  the  SOF  community? 

•  What  can  SOF  leadership,  CLPMs,  and  instructional  supervisors  do  to  address  these  gaps? 

Figure  9.  Gaps  between  Current  State  of  and  Best  Practices  for  Instructional  Supervision 
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Findings  and  Recommendations  for  SOF  Leadership  and  CLPMs 

Many  of  the  instructional  supervisory  behaviors  and  practices  currently  used  in  SOF  1AT  language 
schools  are  comparable  to  those  discussed  in  the  literature  and  those  currently  used  in  external  language 
schools;  however,  some  opportunities  for  improvement  were  identified  in  the  areas  of  instructional 
supervisor  selection,  training,  and  resourcing. 

Selection 

To  support  the  missions  of  SOF  1AT  language  programs,  it  is  important  that  SOF  1AT  leadership  hires  or 
promotes  candidates  with  the  qualifications  necessary  to  be  effective  instructional  supervisors.  This 
includes  selecting  instructional  supervisors  based  on  education,  instructional  experience,  administrative 
experience,  and  English  proficiency. 

Education  and  Experience 

Participants  from  SOF  1AT  language  schools  reported  that  instructional  supervisors  in  their  schools  had  a 
range  of  educational  backgrounds  and  experiences.  In  some  cases,  instructional  supervisors  did  not  have 
degrees  related  to  second  language  acquisition,  education,  or  other  relevant  fields  but  were  hired  or 
promoted  due  to  their  prior  instructional  experience  or  prior  experience  in  the  military  or  SOF 
communities. 

•  Recommendation :  Instructional  supervisors  should  have  the  content  knowledge  necessary  to 
effectively  perform  their  job  responsibilities.  This  includes,  but  is  not  limited  to,  principles  of 
second  language  acquisition,  instructional  methodology,  and  adult  learning.  SOF  1AT  language 
schools  should  hire  or  promote  instructional  supervisors  who  have  Master’s  or  Doctoral  degrees 
in  second  language  acquisition,  education,  or  other  relevant  fields,  or  alternatively,  should  hire  or 
promote  instructional  supervisors  whose  prior  administrative,  instructional,  and  military 
experience  enables  them  to  effectively  perform  their  job  responsibilities  without  formal 
education. 

A  dm  inistrative  Experience 

While  many  participants  from  SOF  1AT  language  schools  reported  that  instructional  supervisors  in  their 
schools  had  administrative  experience,  some  supervisors  did  not  have  this  experience.  It  is  important  that 
instructional  supervisors  have  the  KSAs  necessary  to  perform  not  only  their  classroom  supervisory 
responsibilities,  but  any  administrative  responsibilities  they  may  have  as  well. 

•  Recommendation :  Instructional  supervisors  who  do  not  have  a  background  in  management  or 
administration  should  be  provided  with  training  for  administrative  responsibilities.  Courses  and 
materials  in  the  fields  of  business,  management,  and  educational  leadership  or  educational 
administration  may  be  used  to  develop  instructional  supervisors  with  the  KSAs  necessary  to 
perform  their  administrative  job  responsibilities. 
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English  Proficiency 

In  addition  to  meeting  requirements  for  education  and  experience,  participants  from  SOF  IAT  and 
external  language  schools  indicated  that  instructional  supervisors  need  a  high  degree  of  English 
proficiency,  as  it  is  the  common  language  of  communication  with  instructors  and  stakeholders. 

•  Recommendation-.  Like  instructors,  instructional  supervisors  should  be  proficient  in  English  at  an 
Interagency  Language  Roundtable  (ILR)  Level  of  2  or  higher  ( Special  Operations  Forces 
Instructor  Language  Needs  Assessment  [Technical  Report  #2009010603]).  Task  and  KSA 
analyses  should  be  conducted  to  determine  the  specific  ILR  Level  of  English  proficiency  required 
for  instructional  supervisors  to  be  effective  in  their  roles. 

Training  and  Development 

Even  when  instructional  supervisors  are  highly  qualified  for  their  positions,  there  may  be  areas  in  which 
they  need  additional  training,  such  as  interpersonal  communication  skills  and  the  SOF  IAT  instructional 
context.  In  addition,  instructional  supervisors  need  to  continuously  engage  in  professional  development 
to  stay  current  in  the  field. 

•  Recommendation-.  SOF  IAT  leadership  should  create  an  on-the-job  training  program  for 
instructional  supervisors.  This  training  program  should  be  focused  on  effective  interpersonal 
communication  skills,  such  as  active  listening,  administrative  skills,  and  the  SOF  IAT 
instructional  context.  Additionally,  SOF  IAT  leadership  could  provide  a  module  on  effective 
instructional  supervision  during  CLPM  training.  Courses  and  materials  in  the  fields  of  business, 
management,  and  educational  leadership  or  educational  administration  may  be  leveraged  to  create 
these  trainings. 

•  Recommendation-.  Instructional  supervisors  should  maintain  memberships  in  professional 
organizations  related  to  foreign  languages,  second  language  acquisition,  and  instruction,  such  as 
ACTFL.  In  addition,  instructional  supervisors  should  pursue  professional  development 
opportunities  related  to  management  and  educational  administration. 

•  Recommendation :  SOF  IAT  leadership  should  work  with  SWA  to  develop  a  training  module  on 
Instructor  Feedback  Reports  (IFRs)  for  instructional  supervisors  similar  to  the  IFR  module  for 
CLPM  training.  Objectives  of  an  IFR  training  module  for  instructional  supervisors  should 
include  familiarizing  instructional  supervisors  with  the  Instructor  Evaluation  and  Feedback 
System,  how  the  system  works  and  the  instructional  supervisor’s  role  in  it,  and  how  to  interpret 
an  IFR.  A  training  module  would  ensure  instructional  supervisors  become  familiar  with  the  IFR 
and  the  Instructor  Developmental  Feedback  Guide  and  its  suggestions  so  that  instructional 
supervisors  are  prepared  to  assist  instructors  in  any  ways  necessary  to  improve  in  areas  indicated 
in  the  IFR.  Other  information  that  should  be  included  in  the  module  is  the  process  for  GS 
instructional  supervisors  to  work  with  vendors  to  ensure  contract  instructors  receive  their  IFRs. 

•  Recommendation-.  SOF  IAT  leadership  should  collect  information  about  instructional  supervisors’ 
performance  to  identify  supervisors’  strengths  and  areas  for  improvement.  This  performance 
information  could  be  used  to  create  tools  to  support  supervisors’  continued  development. 


9/28/2012 


©  SWA  Consulting  Inc.,  2012 
Technical  Report  [2012010632] 


Page  46 


SOFLO  Support  Project 


Best  Practices  for  Supervision  of  Instructors 


Resourcing 

Along  with  instructional  supervisor  selection,  training,  and  development,  SOF  1AT  leadership  and 
CLPMs  play  important  roles  in  providing  necessary  resources  to  instructional  supervisions  to  remove 
barriers  to  supervisory  effectiveness  and  enable  supervisors  to  be  more  effective  in  their  roles. 

Supervisor  -  Instructor  Ratio 

As  discussed  in  Section  Ill,  a  psychological  concept  known  as  span  of  attention  states  that  a  person  can 
only  think  about  seven  things  simultaneously  (Entwisle  &  Walton,  1961).  Participants  from  SOF  LAT 
language  schools  reported  that  the  number  of  instructors  per  instructional  supervisor  ranges  from  5-35, 
indicating  that  many  instructional  supervisors  are  supervising  more  instructors  than  is  ideal. 

•  Recommendation-.  SOF  IAT  leadership  should  be  aware  of  instructional  supervisors’  workload 
and  provide  support  staff,  such  as  academic  specialists  and  administrative  personnel,  to  ensure 
supervisors  have  the  resources  they  need  to  perform  all  tasks  efficiently  and  effectively. 
Additionally,  lead  instructors  could  be  appointed  to  assist  with  certain  supervisory  tasks,  such  as 
providing  coaching  and  mentoring  or  resolving  minor  disputes.  As  one  interviewee  said,  “The 
most  successful  supervisor  identifies  strengths  and  weaknesses  and  uses  them  for  the  project.” 

By  utilizing  the  strengths  of  instructors  and  support  staff,  instructional  supervisors  can  reduce 
their  day-to-day  workload  and  make  sure  that  their  time  is  being  used  efficiently  and  effectively, 
to  the  benefit  of  the  instructors  they  supervise. 

Resources  for  Professional  Development 

As  discussed  in  the  training  and  development  section,  professional  development  is  vital  for  instructional 
supervisors  to  stay  informed  on  current  developments  in  second  language  acquisition  and  education. 
However,  instructional  supervisors  in  SOF  IAT  language  schools  may  have  numerous  demands  on  their 
time  and  feel  they  are  unable  to  attend  conferences  and  workshops. 

•  Recommendation-.  SOF  IAT  leadership  should  provide  resources  to  instructional  supervisors  to 
ensure  that  they  have  time  and  funding  necessary  to  engage  in  professional  development. 

•  Recommendation-.  SOF  IAT  leadership  should  ensure  instructional  supervisors  have  adequate 
funding  to  purchase  necessary  resources  requested  by  instructors  and  provide  instructors  with 
professional  development  opportunities. 

Contract  Issues 

Many  participants  reported  difficulties  arising  from  an  inability  to  directly  supervise  contract  instructors. 
While  these  challenges  spanned  a  variety  of  situations,  the  two  that  were  mentioned  most  often  were  the 
inability  to  provide  instructors  with  necessary  training  and  feedback. 

•  Recommendation-.  SOF  IAT  leadership  should  coordinate  with  contract  vendors  to  ensure  contract 
instructors  receive  training  recommended  by  GS  instructional  supervisors.  Also,  SOF  IAT 
leadership  should  ensure  GS  instructional  supervisors  know  the  proper  channels  to  use  to  provide 
contract  instructors  with  feedback.  It  is  also  important  that  existing  resources,  such  as  IFRs,  are 
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delivered  to  contract  instructors  by  their  vendor  supervisors,  as  these  reports  provide  contract 
instructors  with  valuable  feedback  on  their  performance,  particularly  when  GS  instructional 
supervisors  are  unable  to  provide  this  type  of  feedback  directly  to  these  instructors. 

Summary 

In  summary,  SOF  1AT  leadership  and  CLPMs  can  maintain  and  further  develop  effective  instructional 
supervisory  behaviors  and  practices  in  SOF  IAT  language  schools  through  selection,  training  and 
developing,  and  resourcing. 

Findings  and  Recommendations  for  Instructional  Supervisors 

Along  with  SOF  IAT  leadership  and  CLPMs,  current  instructional  supervisors  also  play  a  role  in 
maintaining  and  further  developing  effective  instructional  supervisory  behaviors  and  practices  in  SOF 
IAT  language  schools.  The  recommendations  provided  to  instructional  supervisors  span  the  areas  of 
classroom  supervision  and  other  effective  instructional  supervisory  behaviors  and  skills. 

Classroom  or  Clinical  Supervision 

Observation,  evaluation,  and  feedback  are  important  instructional  supervisory  responsibilities  and  directly 
impact  instructors’  classroom  performance.  Instructional  supervisors  must  gain  instructor  buy-in  in  the 
observation,  evaluation,  and  feedback  processes  to  maximize  the  effectiveness  of  these  processes. 

Observation 

All  participants  from  SOF  IAT  language  schools  reported  that  instructional  supervisors  conduct 
classroom  observations  at  least  once  per  course,  with  a  majority  reporting  they  conduct  either  announced 
or  unannounced  observations  twice  per  course.  However,  few  participants  reported  taking  steps  to  gain 
instructor  buy-in  prior  to  conducting  classroom  observations. 

•  Recommendation :  Pre-observation  conferences  can  be  used  to  gain  instructor  buy-in  in 
observation,  evaluation  and  feedback  processes  and  reassure  instructors  that  the  goal  of  these 
processes  is  to  help  them  improve  their  classroom  performance.  Instructional  supervisors  should 
meet  with  instructors  prior  to  conducting  observations  to  discuss  the  observation  criteria  and  to 
address  any  concerns  instructors  might  have  about  the  observations.  Having  instructors  provide  a 
copy  of  their  lesson  plans  and  asking  if  there  are  any  specific  areas  the  instructor  would  like 
feedback  on  can  help  instructors  feel  involved  in  the  observation  process  and  reduce  the  stress 
that  accompanies  being  observed. 

•  Recommendation :  Instructional  supervisors  and  Quality  Assurance  Surveillance  Program  (QASP) 
personnel  should  use  IFRs  to  identify  instructors  who  could  benefit  the  most  from  receiving 
feedback  based  on  observations  of  their  classroom  performance.  This  information  can  be  used  to 
prioritize  observations  so  instructional  supervisors  and  QASP  personnel  can  use  their  time 
efficiently  and  effectively. 
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Evaluation 

As  previously  discussed,  instructional  supervisors  may  supervise  up  to  35  instructors,  in  addition  to  the 
administrative  responsibilities  they  have.  This  makes  it  difficult  for  instructional  supervisors  to 
frequently  evaluate  all  of  their  instructors.  Infrequent  evaluations  can  cause  instructors  to  feel  the 
evaluations  do  not  fully  represent  their  classroom  instruction  and  are  therefore  inauthentic  and  not  useful 
(Christen  &  Murphy,  1987). 

•  Recommendation :  Instructional  supervisors  should  use  shorter,  more  frequent  observations  (e.g., 
5-15  minutes  once  a  week)  to  provide  a  more  comprehensive  picture  of  each  instructor’s 
performance.  This  approach  may  be  seen  as  less  intrusive  by  instructors  and  students  (Marshall, 
2005).  Additionally,  peer  evaluation  can  be  leveraged  to  provide  instructors  with  more  frequent 
evaluation  without  requiring  additional  time  from  instructional  supervisors  (Christen  &  Murphy, 
1987). 

Feedback 

After  observations  and  evaluations  occurred,  most  participants  from  SOF  1AT  language  schools  reported 
that  instructional  supervisors  initiated  feedback  meetings  with  instructors;  however,  instructors  may 
disregard  this  feedback  if  they  feel  it  reflects  only  a  snapshot  of  their  classroom  performance  (Blumberg, 
2001)  or  if  they  feel  it  is  overly  critical. 

•  Recommendation :  Feedback  should  be  based  on  a  number  of  sources,  including  formal 
observations,  peer  evaluations,  and  student  feedback.  Instructional  supervisors  should  leverage 
existing  resources,  such  as  IFRs  and  the  Instructor  Developmental  Feedback  Guide,  to  provide 
instructors  with  multi-source  feedback  and  recommendations  for  improvement. 

•  Recommendation :  Instructional  supervisors  should  deliver  feedback  in  an  open,  honest,  and 
supportive  manner.  Feedback  should  be  strength-based  and  criticism  should  be  constructive  and 
non-judgmental  (Lee,  2010;  Schonberger,  2001).  Two  models  for  providing  positive  and 
negative  feedback  are  the  Behavior,  Effect,  and  Thank  you  (BET)  and  Behavior,  Effect,  Action, 
and  Result  (BEAR)  models,  respectively  (Flarms  and  Roebuck,  2010).  In  both  cases, 
instructional  supervisors  describe  the  positive  or  negative  behaviors  that  are  affecting 
performance  and  the  effects  these  behaviors  have  on  performance.  In  the  BET  model  for  positive 
feedback,  supervisors  then  thank  instructors  for  their  performance.  In  the  BEAR  model  for 
negative  feedback,  supervisors  then  describe  possible  actions  that  could  be  taken  to  address  issues 
and  the  positive  results  these  actions  could  have,  as  well  as  possible  consequences  of  not  taking 
recommended  actions. 

Other  Effective  Instructional  Supervisory  Behaviors  and  Skills 

Many  factors  contribute  to  the  effectiveness  of  instructional  supervisors  in  both  administrative  and 
classroom  supervisory  behaviors.  Participants  in  SOF  IAT  language  schools  identified  cultural 
understanding  and  effective  interpersonal  communication  skills  as  qualities  that  can  reduce  challenges 
faced  by  instructional  supervisors. 
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Cultural  Understanding 

Participants  from  SOF  IAT  language  schools  indicated  that  cultural  understanding  was  an  important 
qualification  for  instructional  supervisors,  particularly  when  they  supervisors  instructors  from  diverse 
cultural  and  ethnic  backgrounds.  Challenges  related  to  cultural  understanding  reported  by  instructional 
supervisors  at  SOF  IAT  language  programs  include  accommodating  religious  preferences  and  gender 
roles. 

•  Recommendation-.  Instructional  supervisors  who  supervise  instructors  whose  culture  is  different 
than  their  own  should  research  and  make  an  effort  to  understand  these  cultures.  This  might 
include  asking  instructors  to  explain  various  aspects  of  their  culture,  which  could  serve  as  a 
means  of  building  rapport,  in  addition  to  reducing  misunderstandings  due  to  cultural  differences. 
Additionally,  while  it  is  less  important  for  instructional  supervisors  to  be  proficient  in  every 
language  they  supervise,  learning  greetings  and  commonly  used  phrases  in  those  languages  can 
also  help  instructional  supervisors  build  rapport  with  instructors. 

Communication 

Instructional  supervisors  can  contribute  to  effective  communication  and  build  rapport  with  instructors  by 
being  open  and  honest  with  the  instructors  they  supervise. 

•  Recommendation :  Instructional  supervisors  should  strive  to  be  open  and  honest  in  their 
communication  with  instructors.  Supervisors  should  ensure  instructors  know  their  input  is  valued 
by  encouraging  instructors  to  make  suggestions  and  express  concerns  and  by  carefully 
considering  instructors’  input.  Instructional  supervisors  should  also  be  willing  to  admit  their  own 
mistakes  and  shortcomings  and  take  steps  to  address  them,  whether  by  apologizing  for  a  mistake 
or  asking  for  help  when  necessary.  Instructors  may  be  more  likely  to  bring  their  concerns  to 
instructional  supervisors  if  they  feel  that  doing  so  will  not  put  their  jobs  in  jeopardy.  Likewise, 
when  attempting  to  mediate  conflicts  between  instructors,  instructional  supervisors  should  make 
sure  they  have  and  understand  all  of  the  facts  before  making  judgments. 

Summary 

Much  like  SOF  leadership  and  CLPMs,  instructional  supervisors  also  play  important  roles  in  maintaining 
and  further  developing  effective  instructional  supervisory  behaviors  and  practices  in  SOF  IAT  language 
schools.  By  using  the  recommendations  provided  to  instructional  supervisors  in  the  areas  of  classroom 
supervision  and  other  effective  instructional  supervisory  behaviors  and  skills,  these  supervisors  will  be 
more  effective  in  their  roles  and  in  turn,  enable  instructors  to  be  more  effective  in  the  classroom. 

Conclusion 

In  conclusion,  although  instructional  supervisors  in  SOF  IAT  language  schools  currently  engage  in  many 
best  practices  for  effective  instructional  supervision,  SOF  leadership,  CLPMs,  and  instructional 
supervisors  can  use  the  recommendations  provided  to  close  the  gaps  between  the  current  and  desired 
states  of  instructional  supervision  in  SOF  IAT  language  schools.  In  doing  so,  instructional  supervisors 
can,  and  will  continue  to,  play  important  roles  in  increasing  instructor  effectiveness,  thereby  helping  to 
develop  language-capable  SOF  personnel  through  SOF  IAT. 
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ABOUT  SWA  CONSULTING  INC. 

SWA  Consulting  Inc.  (formerly  Surface,  Ward,  and  Associates)  provides  analytics  and  evidence-based 
solutions  for  clients  using  the  principles  and  methods  of  industrial/organizational  (I/O)  psychology.  Since 
1997,  SWA  has  advised  and  assisted  corporate,  non-profit  and  governmental  clients  on: 

•  Training  and  development 

•  Performance  measurement  and  management 

•  Organizational  effectiveness 

•  Test  development  and  validation 

•  Program/training  evaluation 

•  Work/job  analysis 

•  Needs  assessment 

•  Selection  system  design 

•  Study  and  analysis  related  to  human  capital  issues 

•  Metric  development  and  data  collection 

•  Advanced  data  analysis 

One  specific  practice  area  is  analytics,  research,  and  consulting  on  foreign  language  and  culture  in  work 
contexts.  In  this  area,  SWA  has  conducted  numerous  projects,  including  language  assessment  validation 
and  psychometric  research;  evaluations  of  language  training,  training  tools,  and  job  aids;  language  and 
culture  focused  needs  assessments  and  job  analysis;  and  advanced  analysis  of  language  research  data. 

Based  in  Raleigh,  NC,  and  led  by  Drs.  Eric  A.  Surface  and  Stephen  J.  Ward,  SWA  now  employs  close  to 
twenty  I/O  professionals  at  the  masters  and  PhD  levels.  SWA  professionals  are  committed  to  providing 
clients  the  best  data  and  analysis  upon  which  to  make  evidence-based  decisions.  Taking  a  scientist- 
practitioner  perspective,  SWA  professionals  conduct  model-based,  evidence-driven  research  and 
consulting  to  provide  the  best  answers  and  solutions  to  enhance  our  clients’  mission  and  business 
objectives.  SWA  has  competencies  in  measurement,  data  collection,  analytics,  data  modeling,  systematic 
reviews,  validation,  and  evaluation. 

For  more  information  about  SWA,  our  projects,  and  our  capabilities,  please  visit  our  website  (www.swa- 
consulting.com)  or  contact  Dr.  Eric  A.  Surface  ( esurface@s wa-consulting, com)  or  Dr.  Stephen  J.  Ward 
(sward@swa-consulting.com). 

The  following  SWA  Consulting  Inc.  team  members  contributed  to  this  report  (listed  in  alphabetical 
order): 

Ms.  Gwendolyn  Good 

Ms.  Lindsey  Jeralds 

Dr.  Jennifer  Lindberg  McGinnis 

Mr.  Jeff  O’Hallaron 

Dr.  Eric  A.  Surface 
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UNITED  STATES  SPECIAL  OPERATIONS  COMMAND 

7701  TAMPA  POINT  BOULEVARD 

MACDILL  AIR  FORCE  BASE,  FLORIDA  33621-5323 


FMD-J1/7-TL  9  July  201 2 

MEMORANDUM  FOR  SEE  DISTRIBUTION 

SUBJECT :  Request  for  Participation  in  Study  of  Best  Practices  for  Supervising 
Language  Instructors 


1 .  We  are  working  in  conjunction  with  SWA  Consulting  Inc.  (SWA)  to  identify  best 
practices  for  supervision  of  foreign  language  instructors  and  we  need  your  support  in 
collecting  relevant  information  about  existing  instructor  supervision  practices. 

2.  SWA  will  be  contacting  you  to  collect  this  information  in  the  near  future  and  any 
information  you  can  provide  would  be  most  appreciated.  Our  aim  is  to  collect  this 
information  by  July  18,  2012  to  facilitate  publication  of  the  technical  report  by  the  end  of 
the  Fiscal  Year. 

3.  To  indicate  your  interest  in  participating  in  this  effort,  please  visit  the  following  link  to 
complete  a  short  survey: 

Click  the  link  to  take  the  survey  or  paste  it  into  your  browser: 
http://swa.us2.qualtrics.com/SE/?SID=SV  6GAXDQAbHiTfTWQ. 

4.  If  you  have  any  questions  about  the  study,  please  contact  Dr.  Jenn  McGinnis  from 
SWA  Consulting  Inc.  at  (919)  835-1562  ext.  7007,  jmcginnis@swa-consulting.com  or 
me  atjohn.donnelly@socom.mil,  (813)  826-6040. 

Thank  you  in  advance  for  your  time  and  consideration. 


DAFC,  Directorate  of  Force  Management  and 
Development,  J1/7-TL 
Director,  SOF  Language  Office 


DISTRIBUTION: 

LTC  Curtis  W.  Price 

U.S.  Army  John  F.  Kennedy  Special  Warfare  Center  and  School  (SWCS) 
Basic  Language  Course  (BLC)  and  Intermediate  Language  Course  (ILC) 
Ft.  Bragg,  NC  28307 


FMS-J1/7-TL 

SUBJECT :  Request  for  Participation  in  Study  of  Best  Practices  for  Supervising  Language 
Instructors 


Mr.  Terry  Schnurr 

SWCS  ILC 

Ft.  Bragg,  NC  28307 

Dr.  Marla  D.  Federe 

Naval  Special  Warfare  Advanced  Training  Command 
Naval  Base  Coronado,  CA  92135 

Ms.  Alicia  N.  Spurling 

Air  Force  Special  Operations  Command 

Hurlburt  Field,  FL  32544 

Ms.  Grisel  A.  Mundo 

Air  Force  Special  Operations  Training  Center 
Hurlburt  Field,  FL  32544 

Ms.  Tanya  G.  Woodcook 

U.S.  Marine  Corps  Forces  Special  Operations  Command 
Camp  Lejeune,  NC  28542 

Mr.  Todd  J.  Amis 

Marine  Special  Operations  School 
Camp  Lejeune,  NC  28542 
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UNITED  STATES  SPECIAL  OPERATIONS  COMMAND 

7701  TAMPA  POINT  BOULEVARD 
MACDILL  AIR  FORCE  BASE,  FLORIDA  33621-5323 


12  July  2012 

Director,  Special  Operations  Forces  Language  Office 


[NAME] 

[JOB  TITLE] 

[ORGANIZATION] 

[ADDRESS] 

Dear  [NAME]: 

My  name  is  Jack  Donnelly,  and  I  work  for  the  United  States  Special  Operations 
Command  (USSOCOM)  as  the  Director  of  the  Special  Operations  Forces  Language 
Office  (SOFLO).  This  office  is  currently  working  in  conjunction  with  SWA  Consulting 
Inc.  (SWA)  to  establish  general  best  practices  for  supervision  of  foreign  language 
instructors.  This  research  will  be  beneficial  to  all  parties  who  deal  with  foreign  language 
instruction  or  training,  regardless  of  learner  population.  To  inform  work  on  this  project, 
SOFLO  is  assisting  SWA  in  gathering  information  about  existing  institutional  instructor 
supervision  practices.  These  practices  will  serve  as  a  baseline  for  research  we  are 
conducting  on  best  practices. 

I  am  requesting  your  assistance  in  this  project  by  providing  SWA  with  information  about 
practices  for  the  supervision  of  instructors  used  in  your  organization.  SWA  will  begin 
collecting  information  soon,  and  any  information  you  can  provide  would  be  most 
appreciated.  Our  aim  is  to  collect  this  information  by  July  23,  201 2. 

To  indicate  your  interest  in  participating  in  this  effort,  please  visit  the  following  link  to 
complete  a  short  survey: 

Click  here  to  take  the  survey  or  paste  the  following  URL  into  your  browser: 
http://swa.us2.qualtrics.com/SE/?SID=SV  6GAXDQAbHiTfTWQ 

Please  Note.  This  project  is  sponsored  by  SOFLO  and  USSOCOM  and  is  conducted 
under  a  subcontract  with  CACI  International  Inc.  as  the  prime  and  SWA  Consulting  Inc. 
as  the  technical  lead  subcontractor  (Subcontract  #  B1 1  -1 1 4482;  Prime  #  H92222-1 0-D- 
0017/0007).  This  is  not  a  solicitation,  and  participation  in  no  way  obligates  the 
government  to  contract  with  your  organization.  The  feedback  received  from  your 
organization  will  be  completely  voluntary  on  your  part  and  no  payment  will  be  provided 
by  any  parties  associated  with  this  contract. 


If  you  have  any  questions  about  the  study,  please  contact  Dr.  Jenn  McGinnis  from  SWA 
Consulting  Inc.  at  (919)  835-1562  ext.  7007,  jmcginnis@swa-consulting.com  or  me  at 
john.donnelly@socom.mil,  (813)  826-6040. 

Thank  you  in  advance  for  your  time  and  consideration. 

Sincerely, 

m 

Jack  Donnelly 

Department  of  the  Air  Force  Civilian 
Director,  SOF  Language  Office 
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UNITED  STATES  SPECIAL  OPERATIONS  COMMAND 

7701  TAMPA  POINT  BOULEVARD 
MACDILL  AIR  FORCE  BASE,  FLORIDA  33621-5323 


10  July  2012 

Director,  Special  Operations  Forces  Language  Office 


[NAME] 

[JOB  TITLE] 

[ORGANIZATION] 

[ADDRESS] 

Dear  [NAME]: 

My  name  is  Jack  Donnelly,  and  I  work  for  the  United  States  Special  Operations 
Command  (USSOCOM)  as  the  Director  of  the  Special  Operations  Forces  Language 
Office  (SOFLO).  This  office  is  currently  working  in  conjunction  with  SWA  Consulting 
Inc.  (SWA)  to  establish  general  best  practices  for  supervision  of  foreign  language 
instructors.  This  research  will  be  beneficial  to  all  parties  who  deal  with  foreign  language 
instruction  or  training,  regardless  of  learner  population.  To  inform  work  on  this  project, 
SOFLO  is  assisting  SWA  in  gathering  information  about  existing  institutional  instructor 
supervision  practices.  These  practices  will  serve  as  a  baseline  for  research  we  are 
conducting  on  best  practices. 

I  am  requesting  your  assistance  in  this  project  by  providing  SWA  with  information  about 
practices  for  the  supervision  of  instructors  used  in  your  organization.  SWA  will  begin 
collecting  information  soon,  and  any  information  you  can  provide  would  be  most 
appreciated.  Our  aim  is  to  collect  this  information  by  July  20,  201 2. 

To  indicate  your  interest  in  participating  in  this  effort,  please  visit  the  following  link  to 
complete  a  short  survey: 

Click  here  to  take  the  survey  or  paste  the  following  URL  into  your  browser: 
http://swa.us2.qualtrics.com/SE/?SID=SV  6GAXDQAbHiTfTWQ 

Please  Note.  This  project  is  sponsored  by  SOFLO  and  USSOCOM  and  is  conducted 
under  a  subcontract  with  CACI  International  Inc.  as  the  prime  and  SWA  Consulting  Inc. 
as  the  technical  lead  subcontractor  (Subcontract  #  B1 1  -1 1 4482;  Prime  #  H92222-1 0-D- 
0017/0007).  This  is  not  a  solicitation,  and  participation  in  no  way  obligates  the 
government  to  contract  with  your  organization.  The  feedback  received  from  your 
organization  will  be  completely  voluntary  on  your  part  and  no  payment  will  be  provided 
by  any  parties  associated  with  this  contract. 


If  you  have  any  questions  about  the  study,  please  contact  Dr.  Jenn  McGinnis  from  SWA 
Consulting  Inc.  at  (919)  835-1562  ext.  7007,  jmcginnis@swa-consulting.com  or  me  at 
john.donnelly@socom.mil,  (813)  826-6040. 

Thank  you  in  advance  for  your  time  and  consideration. 

Sincerely, 

m 

Jack  Donnelly 

Department  of  the  Air  Force  Civilian 
Director,  SOF  Language  Office 
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APPENDIX  D:  INTERVIEW  PROTOCOL  FOR  SOF  IAT  LANGUAGE  PROGRAMS 
Greeting 

Hi,  this  is _ calling  from  SWA  Consulting  Inc.  How  are  you  today? 

Introductory  Statement 

During  our  interview  today,  we  will  be  asking  you  some  questions  about  supervision  of  foreign  language 
instructors  in  your  organization.  SWA  Consulting  Inc.  (SWA)  is  currently  working  in  conjunction  with 
the  Special  Operations  Forces  Language  Office  (SOFLO)  to  establish  general  best  practices  for 
supervision  of  foreign  language  instructors.  This  research  will  be  beneficial  to  all  parties  who  deal  with 
foreign  language  instruction  or  training,  regardless  of  learner  population.  Information  that  you  provide 
will  serve  as  a  baseline  for  our  research. 

The  questions  we  will  be  asking  will  take  about  one  hour,  and  will  center  on  different  areas  of 
instructional  supervision  in  your  organization.  We  will  focus  on  current  instructional  supervision 
practices  used  in  your  organization  and  will  be  asking  about  the  position  of  instructional  supervisor, 
typical  relationships  between  supervisors  and  instructors,  strategies  for  effective  supervision  of 
instructors,  and  classroom  observations  and  assessments.  As  you  answer  these  questions,  we  will  be 
taking  notes. 

The  information  you  provide  will  be  kept  confidential  and  will  be  used  for  the  purposes  of  producing  a 
report  for  Mr.  John  Donnelly,  Director  of  SOFLO.  This  report  will  be  released  to  him  and  will  include 
only  summary  results.  The  information  you  provide  will  not  be  attached  to  or  associated  with  your  name 
or  organization.  Instead,  we  will  describe  and  summarize  information  across  multiple  participating 
organizations.  In  addition,  the  information  you  provide  will  be  integrated  with  a  systematic  review  of  the 
literature  on  this  topic,  as  well  as  appropriate  empirical  results  available  from  SWA’s  other  research  and 
evaluation  efforts.  Under  Title  32,  §219  of  the  Code  of  Federal  Regulations  from  the  Department  of 
Defense  Instruction  3216.02,  this  study  qualifies  as  exempt  because  the  information  you  and  other 
interviewees  provide  will  not  be  published  showing  how  specific  individuals  responded,  identities  will  be 
protected,  and  groupings  of  responses  by  external  organization  will  not  be  shown. 

Do  you  have  any  questions  for  us  before  we  begin? 

Interview  Questions 

Position  of  Instructional  Supervisor 

The  first  set  of  questions  we  have  for  you  will  ask  you  about  the  position  of  instructional  supervisor  and 
some  of  the  responsibilities  these  supervisors  have. 

1.  How  many  instructional  supervisors  are  there  in  your  language  school  or  department? 

a.  If  this  role  did  not  exist,  where  would  these  responsibilities  be  redistributed? 

2.  To  whom  do  instructional  supervisors  report? 

a.  Who  are  they  accountable  to/who  manages  them? 
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3.  Who  decides  on  an  instructional  supervisor’s  responsibilities? 

4.  What  are  instructional  supervisors’  responsibilities? 

[If  not  described  above  in  response  to  previous  questions ]  Do  instructional  supervisors: 

a.  Interview  and/or  hire  instructors? 

b.  Train,  mentor,  and/or  orient  instructors? 

c.  Monitor  instructors’  performance? 

d.  Provide  incentives/rewards  to  instructors? 

e.  Provide  constructive  feedback  to  instructors? 

f.  Provide  resources  to  instructors? 

g.  Schedule  courses? 

h.  Select  or  specify  curriculum? 

i.  Develop  curriculum? 

j.  Lead  meetings? 

5.  What  is  the  typical  background  of  an  instructional  supervisor? 

a.  Were  instructional  supervisors  once  instructors? 

b.  Were  instructional  supervisors  once  administrators? 

c.  Are  instructional  supervisors  predominantly  hired  externally  or  promoted  from  within? 

6.  What  are  the  degree  or  education  requirements  to  be  a  supervisor? 

a.  Is  there  a  minimum  requirement  for  years  of  experience  (or  years  instructing)? 

b.  Have  candidates  for  this  position  generally  taken  additional  training  or  certification  to  be 
a  supervisor? 

c.  Is  there  on-the-job  training  for  supervisors? 

7.  What  are  the  most  important  characteristics  of  a  successful  supervisor  (i.e.,  what  is  it  most 
important  that  they  be  good  at)? 

8.  How  important  is  content  knowledge  (i.e.,  knowledge  of  language  instruction)  for  a  supervisor  to 
be  effective  in  his/her  role? 

a.  How  important  is  knowledge  of  the  target  language  and  culture? 

b.  How  important  is  English  proficiency  for  the  supervisor?  For  the  instructors? 
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9.  Across  your  experience,  give  an  example  of  a  particularly  effective  supervisor.  What  did  that 
supervisor  do  that  made  him/her  particularly  effective? 

10.  Across  your  experience,  give  an  example  of  a  particularly  ineffective  supervisor.  What  did  that 
supervisor  do  that  made  him/her  particularly  ineffective? 

Relationships  between  Instructional  Supervisors  and  Instructors 

The  next  few  questions  are  about  the  relationships  between  instructional  supervisors  and  the  instructors 
they  supervise. 

11.  How  many  instructors  does  each  supervisor  supervise? 

12.  What  types  of  instructors  does  each  supervisor  supervise  (i.e.,  GS,  contractor)?  How  would  you 
manage  these  types  of  instructors  differently? 

13.  How  much  time  do  instructional  supervisors  spend  one-on-one  with  instructors  on  a  weekly 
basis? 

14.  What  are  some  of  the  challenges  supervisors  face  in  working  with  instructors? 

Strategies  and  Methods  for  Instructional  Supervision 

In  this  next  set  of  questions,  we  will  ask  you  about  strategies  and  methods  instructional  supervisors  can 
use  to  be  more  effective. 

Resources 

15.  What  power  or  resources  do  supervisors  have  to  help  instructors  improve? 

16.  What  resources  can  supervisors  direct  instructors  to?  How  do  supervisors  find  out  about  these 
resources? 

Models  of  Instruction 

17.  Do  supervisors  have  a  model  of  instruction  they  recommend  to  instructors? 

a.  If  so,  how  was  that  model  developed? 

Performance  Standards 

18.  How  are  performance  standards  for  instructors  decided  upon? 

19.  How  does  the  supervisor  make  the  standards  for  performance  clear  to  instructors? 

Motivation 

20.  What  strategies  are  used  to  motivate  instructors?  Do  these  vary  across  supervisors  or  across 
instructors? 

Professional  Development 
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21.  How  do  supervisors  gain  and  maintain  knowledge  of  current  best  practices  for  instruction? 
Classroom  Observations  and  Assessments 

In  the  following  set  of  questions,  we  will  be  asking  you  about  observations  and  assessments  that 
instructional  supervisors  do  in  the  classroom.  For  the  purposes  of  this  interview,  the  words  “observation” 
and  “assessment”  will  be  used  interchangeably. 

22.  Who  conducts  classroom  observations? 

a.  If  supervisors,  when  do  they  do  classroom  observations? 

b.  How  often? 

c.  Are  they  scheduled  or  unscheduled?  (Are  instructors  forewarned?) 

d.  How  long  are  they? 

23.  Please  give  a  detailed  explanation  of  the  classroom  observation  process. 

a.  Is  there  a  “pre”  meeting  to  involve  instructors  in  developing  the  rubric  for  assessment? 
How  is  buy-in  from  instructors  gained? 

b.  Are  observations  always  done  by  the  same  supervisor  (or  individual)? 

24.  Do  supervisors  equally  distribute  attention  among  both  high-  and  low-  performing  instructors? 

25.  What  tools  do  supervisors  use  for  observation  and  assessment?  How  were  these  developed? 

What  resources  do  supervisors  use  to  create  and  update  them? 

26.  How  much  input  is  sought  from  instructors  in  the  observation/assessment  process? 

27.  What  occurs  after  an  observation  is  conducted  or  an  instructor  is  assessed? 

Closing  Statement 

That  concludes  the  questions  we  have  for  you.  Thank  you  for  your  time  and  input.  The  information  you 
provided  is  invaluable  to  our  goal  of  establishing  general  best  practices  for  supervision  of  foreign 
language  instructors.  If  you  have  any  questions  for  us  later,  please  feel  free  to  call  or  email. 
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APPENDIX  E:  INTERVIEW  PROTOCOL  FOR  EXTERNAL  LANGUAGE  PROGRAMS 
Greeting 

Hi,  this  is _ calling  from  SWA  Consulting  Inc.  How  are  you  today? 

Introductory  Statement 

During  our  interview  today,  we  will  be  asking  you  some  questions  about  supervision  of  foreign  language 
instructors  in  your  organization.  SWA  Consulting  Inc.  (SWA)  is  currently  working  in  conjunction  with 
the  Special  Operations  Forces  Language  Office  (SOFLO)  to  establish  general  best  practices  for 
supervision  of  foreign  language  instructors.  This  research  will  be  beneficial  to  all  parties  who  deal  with 
foreign  language  instruction  or  training,  regardless  of  learner  population.  Information  that  you  provide 
will  serve  as  a  baseline  for  our  research. 

The  questions  we  will  be  asking  will  take  about  one  hour,  and  will  center  on  different  areas  of 
instructional  supervision  in  your  organization.  The  first  two  sets  of  questions  will  focus  on  the  language 
learning  environment  and  language  learners  in  your  organization.  We  will  then  focus  on  current 
instructional  supervision  practices  used  in  your  organization  and  will  be  asking  about  the  position  of 
instructional  supervisor,  typical  relationships  between  supervisors  and  instructors,  strategies  for  effective 
supervision  of  instructors,  and  classroom  observations  and  assessments.  As  you  answer  these  questions, 
we  will  be  taking  notes. 

The  information  you  provide  will  be  kept  confidential  and  will  be  used  for  the  purposes  of  producing  a 
report  for  Mr.  John  Donnelly,  Director  of  SOFLO.  This  report  will  be  released  to  him  and  will  include 
only  summary  results.  The  information  you  provide  will  not  be  attached  to  or  associated  with  your  name 
or  organization.  Instead,  we  will  describe  and  summarize  information  across  multiple  participating 
organizations.  In  addition,  the  information  you  provide  will  be  integrated  with  a  systematic  review  of  the 
literature  on  this  topic,  as  well  as  appropriate  empirical  results  available  from  SWA’s  other  research  and 
evaluation  efforts.  Under  Title  32,  §219  of  the  Code  of  Federal  Regulations  from  the  Department  of 
Defense  Instruction  3216.02,  this  study  qualifies  as  exempt  because  the  information  you  and  other 
interviewees  provide  will  not  be  published  showing  how  specific  individuals  responded,  identities  will  be 
protected,  and  groupings  of  responses  by  external  organization  will  not  be  shown. 

Do  you  have  any  questions  for  us  before  we  begin? 

Interview  Questions 

Language  Learning  Environment 

The  first  set  of  questions  we  have  for  you  is  about  the  language  learning  environment. 

1 .  What  languages  are  taught? 

2.  What  is  the  course  duration  and  total  contact  hours? 

a.  Are  there  differences  in  course  duration  across  languages? 

3.  How  often  do  classes  meet? 
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4.  What  is  the  purpose  for  learning  the  language? 

a.  Is  the  course  required? 

b.  In  your  opinion,  how  motivated  are  the  students  to  learn  the  language? 

5.  What  are  the  targeted  proficiency  levels  or  expected  outcomes  of  language  learning? 

6.  How  is  the  students’  proficiency  assessed?  How  often  is  it  assessed? 

Language  Learners 

For  this  second  set  of  questions,  we  will  be  asking  about  the  students  participating  in  foreign  language 
learning  in  your  program. 

7.  What  is  the  average  age  of  the  students  in  this  language  learning  program?  What  range  of  ages 
do  you  typically  see  in  this  language  learning  program? 

8.  What  is  the  gender  of  most  of  the  students  here? 

9.  In  what  industry  are  most  of  the  students  employed?  What  are  some  of  the  most  common  job 
titles  they  hold? 

10.  Is  the  native  language  of  most  students  English?  What  other  native  languages  have  past  students 
had? 

11.  Do  most  of  the  students  have  previous  language  learning  experience  (i.e.,  high  school  or  college 
courses,  other  language  learning  programs)? 

12.  What  is  the  initial  proficiency  level  of  most  of  the  incoming  students? 

a.  Is  there  screening  or  selection  that  takes  place?  Can  anyone  who  signs  up  take  the 
course,  or  must  they  meet  certain  requirements  (e.g.,  experience,  aptitude)? 

b.  Is  there  a  placement  process  that  takes  place?  That  is,  are  students  placed  in  a  particular 
language?  If  so,  what  criteria  are  used  to  place  them  in  a  particular  language? 

Position  of  Instructional  Supervisor 

This  next  set  of  questions  we  have  asks  about  the  position  of  instructional  supervisor  and  some  of  the 
responsibilities  these  supervisors  have. 

13.  How  many  instructional  supervisors  are  there  in  your  language  school  or  department? 

a.  If  this  role  did  not  exist,  where  would  these  responsibilities  be  redistributed? 

14.  To  whom  do  instructional  supervisors  report? 

a.  Who  are  they  accountable  to/who  manages  them? 

15.  Who  decides  on  an  instructional  supervisor’s  responsibilities? 
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16.  What  are  instructional  supervisors’  responsibilities? 

[If  not  described  above  in  response  to  previous  questions ]  Do  instructional  supervisors: 

a.  Interview  and/or  hire  instructors? 

b.  Train,  mentor,  and/or  orient  instructors? 

c.  Monitor  instructors’  performance? 

d.  Provide  incentives/rewards  to  instructors? 

e.  Provide  constructive  feedback  to  instructors? 

f.  Provide  resources  to  instructors? 

g.  Schedule  courses? 

h.  Select  or  specify  curriculum? 

i.  Develop  curriculum? 

j.  Lead  meetings? 

17.  What  is  the  typical  background  of  an  instructional  supervisor? 

a.  Were  instructional  supervisors  once  instructors? 

b.  Were  instructional  supervisors  once  administrators? 

c.  Are  instructional  supervisors  predominantly  hired  externally  or  promoted  from  within? 

18.  What  are  the  degree  or  education  requirements  to  be  a  supervisor? 

a.  Is  there  a  minimum  requirement  for  years  of  experience  (or  years  instructing)? 

b.  Have  candidates  for  this  position  generally  taken  additional  training  or  certification  to  be 
a  supervisor? 

c.  Is  there  on-the-job  training  for  supervisors? 

19.  What  are  the  most  important  characteristics  of  a  successful  supervisor  (i.e.,  what  is  it  most 
important  that  they  be  good  at)? 

20.  How  important  is  content  knowledge  (i.e.,  knowledge  of  language  instruction)  for  a  supervisor  to 
be  effective  in  his/her  role? 

a.  How  important  is  knowledge  of  the  target  language  and  culture? 

b.  How  important  is  English  proficiency  for  the  supervisor?  For  the  instructors? 
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21.  Across  your  experience,  give  an  example  of  a  particularly  effective  supervisor.  What  did  that 
supervisor  do  that  made  him/her  particularly  effective? 

22.  Across  your  experience,  give  an  example  of  a  particularly  ineffective  supervisor.  What  did  that 
supervisor  do  that  made  him/her  particularly  ineffective? 

Relationships  between  Instructional  Supervisors  and  Instructors 

The  next  few  questions  are  about  the  relationships  between  instructional  supervisors  and  the  instructors 
they  supervise. 

23.  How  many  instructors  does  each  supervisor  supervise? 

24.  What  types  of  instructors  does  each  supervisor  supervise  (i.e.,  GS,  contractor)?  How  would  you 
manage  these  types  of  instructors  differently? 

25.  How  much  time  do  instructional  supervisors  spend  one-on-one  with  instructors  on  a  weekly 
basis? 

26.  What  are  some  of  the  challenges  supervisors  face  in  working  with  instructors? 

Strategies  and  Methods  for  Instructional  Supervision 

In  this  next  set  of  questions,  we  will  ask  you  about  strategies  and  methods  instructional  supervisors  can 
use  to  be  more  effective. 

Resources 

27.  What  power  or  resources  do  supervisors  have  to  help  instructors  improve? 

28.  What  resources  can  supervisors  direct  instructors  to?  How  do  supervisors  find  out  about  these 
resources? 

Models  of  Instruction 

29.  Do  supervisors  have  a  model  of  instruction  they  recommend  to  instructors? 

a.  If  so,  how  was  that  model  developed? 

Performance  Standards 

30.  How  are  performance  standards  for  instructors  decided  upon? 

31.  How  does  the  supervisor  make  the  standards  for  performance  clear  to  instructors? 

Motivation 

32.  What  strategies  are  used  to  motivate  instructors?  Do  these  vary  across  supervisors  or  across 
instructors? 

Professional  Development 
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33.  How  do  supervisors  gain  and  maintain  knowledge  of  current  best  practices  for  instruction? 
Classroom  Observations  and  Assessments 

In  this  final  set  of  questions,  we  will  be  asking  you  about  observations  and  assessments  that  instructional 
supervisors  do  in  the  classroom.  For  the  purposes  of  this  interview,  the  words  “observation”  and 
“assessment”  will  be  used  interchangeably. 

34.  Who  conducts  classroom  observations? 

a.  If  supervisors,  when  do  they  do  classroom  observations? 

b.  How  often? 

c.  Are  they  scheduled  or  unscheduled?  (Are  instructors  forewarned?) 

d.  How  long  are  they? 

35.  Please  give  a  detailed  explanation  of  the  classroom  observation  process. 

a.  Is  there  a  “pre”  meeting  to  involve  instructors  in  developing  the  rubric  for  assessment? 
How  is  buy-in  from  instructors  gained? 

b.  Are  observations  always  done  by  the  same  supervisor  (or  individual)? 

36.  Do  supervisors  equally  distribute  attention  among  both  high-  and  low-  performing  instructors? 

37.  What  tools  do  supervisors  use  for  observation  and  assessment?  How  were  these  developed? 
What  resources  do  supervisors  use  to  create  and  update  them? 

38.  How  much  input  is  sought  from  instructors  in  the  observation/assessment  process? 

39.  What  occurs  after  an  observation  is  conducted  or  an  instructor  is  assessed? 

Closing  Statement 

That  concludes  the  questions  we  have  for  you.  Thank  you  for  your  time  and  input.  The  information  you 
provided  is  invaluable  to  our  goal  of  establishing  general  best  practices  for  supervision  of  foreign 
language  instructors.  If  you  have  any  questions  for  us  later,  please  feel  free  to  call  or  email. 
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APPENDIX  F:  INTERVIEW  ANALYSIS  TECHNICAL  APPENDIX 

Data  analysis  was  needed  to  identify  themes  from  the  interviews  conducted  with  SOF  1AT  language 
program  administrators,  SOF  1AT  language  program  vendors,  and  administrators  at  language  schools 
external  to  the  SOF  1AT  community.  Data  analysis  began  with  coding  the  interview  transcripts  with 
structural  and  content  codes.  To  ensure  the  quality  of  the  analysis,  two  coders  were  used  and  intercoder 
agreement  was  calculated.  The  prevalence  of  themes  was  then  identified  by  frequency  counts. 

Interview  Transcript  Analysis 

The  interview  transcripts  were  exported  from  Microsoft  Word  into  Microsoft  Excel,  and  the  data  from  the 
interviews  were  coded  in  two  phases:  (1)  structural  coding,  and  (2)  content  coding.  Each  phase  provided 
a  more  detailed  analysis  of  the  data. 

Structural  Coding  Phase 

Structural  coding  works  well  for  data  collected  through  discrete  questions  and  probes  (e.g.,  interviews) 
that  are  repeated  across  multiple  files  in  a  data  set  (MacQueen,  McLellan-Lemal,  Bartholow,  &  Milstein, 
2008;  Saldana,  2009).  Structural  coding  makes  “subsequent  analyses  easier  by  identifying  all  of  the  text 
associated  with  a  particular  question  and  its  associated  probes”  (MacQueen  et  al.,  2008,  p.  124).  This  step 
is  essential  to  prepare  for  the  more  detailed  content  coding  phase. 

Structural  codes  were  developed  using  the  interview  protocols  for  the  SOF  1AT  language  schools  and 
external  language  schools  (Appendices  D  and  E,  pp.  64-72).  Across  the  two  interview  protocols,  each 
question  was  assigned  a  number.  Within  each  segment,  the  question  asked  by  the  interviewer  and  the 
complete  response  provided  by  the  interviewee(s)  were  included.  Additionally,  any  dialog  between  the 
interviewer  and  the  interviewee(s)  resulting  from  the  initial  question  was  captured  in  the  segment.  Each 
complete  interview  question-and-answer  segment  was  exported  from  Microsoft  Word  into  Microsoft 
Excel,  and  the  appropriate  structural  code  (i.e.,  question  number)  was  applied.  The  structural  codes  were 
primarily  used  to  segment  the  interview  data  for  the  codebook  development  and  content  coding  phases. 

Codebook  Development  Phase 

When  coding  qualitative  data,  codebooks  and  coding  instructions  are  developed  to  create  a  shared  mental 
model  among  coders  for  applying  the  comment  codes.  By  establishing  a  detailed  guideline  for  applying 
content  codes,  the  interchangeability  of  coders  is  enhanced  (i.e.,  a  new  coder  could  be  added  to  the 
process  at  any  time  without  affecting  the  coding).  Furthermore,  coding  instructions  allow  others  to 
replicate  the  results  of  the  study  or  to  apply  the  same  codes  to  additional  data  collected  at  a  later  date. 

To  create  content  codes  for  this  project,  text  segments  assigned  to  each  structural  code  were  reviewed  for 
recurring  themes.  Two  research  team  members  independently  created  themes  and  then  met  to  ensure  each 
theme  was  represented  and  to  further  clarify  any  unclear  themes.  Then  the  coders  developed  rules  about 
when  the  codes  should  and  should  not  be  applied,  in  accordance  with  established  guidelines  (MacQueen, 
McLellan,  Kay,  &  Milstein,  1998).  The  researchers  conducted  a  final  review  of  the  codes  for  clarity  and 
conciseness. 
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Content  Coding  Phase 

Content  coding  further  identifies  more  specific  themes  that  emerge  from  respondents’  answers  to  the 
interview  questions.  Each  discrete  segment  of  text  identified  during  structural  coding  is  content  coded. 
Although  it  is  preferable  if  a  single  code  can  be  applied  to  a  single  unit  of  text,  simultaneous  coding  (i.e., 
applying  multiple  codes  to  a  single  unit  of  text)  is  often  necessary  with  complex  topics  and  discussion. 
This  form  of  coding  can  be  used  when  it  is  impossible  to  capture  the  sentiment  of  the  text  with  a  single 
code  (Saldana,  2009). 

Coders  began  the  content  coding  process  by  reviewing  the  codebook  and  making  any  necessary 
modifications.  The  segments  were  then  independently  coded.  Within  the  content  coding  phase, 
simultaneous  coding  was  used  to  accommodate  the  need  to  describe  specific  segments  using  multiple 
codes. 

Quality  Assurance  Phase 

During  content  coding,  coder  training  and  the  use  of  multiple  coders  were  used  to  enhance  the  quality  of 
analysis. 

Prior  to  content  coding,  two  coders  were  trained  to  ensure  that  each  coder  understood  and  executed  the 
same  process.  A  training  session  included  discussions  about  guidelines  for  coding  and  how  and  when  to 
calculate  intercoder  agreement  during  the  content  coding  process.  Two  coders  worked  independently  to 
assign  content  codes  to  segments  of  text  and  then  met  to  review  disagreements  and  discuss  to  consensus. 

Absolute  agreement  was  calculated  for  the  content  coding  phase.  This  method  assessed  intercoder 
agreement  by  calculating  the  percentage  of  segments  that  the  coders  agreed  upon  (Bernard  &  Ryan, 
2010).  Higher  levels  of  absolute  agreement  were  an  indication  the  coders  were  applying  the  codes 
consistently.  The  percentage  of  absolute  agreement  for  the  content  coding  phase  was  74%.  The  coders 
discussed  their  coding  disagreements  to  100%  agreement. 
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